-- Logs begin at Thu 2022-10-27 12:46:36 UTC, end at Thu 2022-10-27 13:43:17 UTC. -- Oct 27 13:10:02.637451 dv-cinder-862432-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 27 13:10:02.649706 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 27 13:10:02 2022] *** Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: nodename: dv-cinder-862432-3-iscsi Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: machine: x86_64 Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: clock source: unix Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: pcre jit disabled Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: detected number of CPU cores: 4 Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: current working directory: / Oct 27 13:10:02.651717 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: detected binary path: /usr/bin/uwsgi-core Oct 27 13:10:02.652383 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: your processes number limit is 47680 Oct 27 13:10:02.652383 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: your memory page size is 4096 bytes Oct 27 13:10:02.652383 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: detected max file descriptor number: 2048 Oct 27 13:10:02.652383 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: lock engine: pthread robust mutexes Oct 27 13:10:02.652383 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: thunder lock: enabled Oct 27 13:10:02.652383 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 27 13:10:02.652383 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 27 13:10:02.690833 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: Python main interpreter initialized at 0x55839fe169c0 Oct 27 13:10:02.690833 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: python threads support enabled Oct 27 13:10:02.690833 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: your server socket listen backlog is limited to 100 connections Oct 27 13:10:02.690833 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: your mercy for graceful operations on workers is 90 seconds Oct 27 13:10:02.691171 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: mapped 403077 bytes (393 KB) for 2 cores Oct 27 13:10:02.691171 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: *** Operational MODE: preforking *** Oct 27 13:10:02.691430 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: *** uWSGI is running in multiple interpreter mode *** Oct 27 13:10:02.691430 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: spawned uWSGI master process (pid: 81429) Oct 27 13:10:02.691612 dv-cinder-862432-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 27 13:10:02.693175 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: spawned uWSGI worker 1 (pid: 81441, cores: 1) Oct 27 13:10:02.693175 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: spawned uWSGI worker 2 (pid: 81442, cores: 1) Oct 27 13:10:02.693175 dv-cinder-862432-3-iscsi devstack@keystone.service[81429]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 27 13:10:06.946587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 13:10:06.946587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: from cryptography.utils import int_from_bytes Oct 27 13:10:06.947341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 13:10:06.947341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: from cryptography.utils import int_from_bytes Oct 27 13:10:06.947746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 13:10:06.947746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: from cryptography.utils import int_from_bytes Oct 27 13:10:06.947845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 27 13:10:06.947845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: from cryptography.utils import int_from_bytes Oct 27 13:10:06.996563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 27 13:10:06.996699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 27 13:10:06.996803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 27 13:10:06.996892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 27 13:10:06.996968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] command line args: [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 27 13:10:06.997042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] command line args: [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 27 13:10:06.997189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 27 13:10:06.997292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 27 13:10:06.997363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ================================================================================ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 27 13:10:06.997495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ================================================================================ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 27 13:10:06.997584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] admin_token = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.997713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] admin_token = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.997816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config_dir = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.997953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config_dir = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] config_source = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] config_source = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] debug = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] debug = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998572 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.998937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:06.999954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_config_append = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_config_append = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_dir = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_dir = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.000898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.001958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.002981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.003968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.004393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.004492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.004574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] publish_errors = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.004640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] publish_errors = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.004792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.004874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] standard_threads = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.005980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] standard_threads = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_journal = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_journal = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.006932 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_json = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_json = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_stderr = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_stderr = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] use_syslog = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] use_syslog = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007422 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 27 13:10:07.007673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.007748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.007906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.007988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.008175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.008262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.008454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.008528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.008704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.008793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.008937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.009978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010236 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.010944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.011905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.012887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.013071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.013151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.013574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.013657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.013819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.013892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.014979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.015987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.016965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017419 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.017869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.018932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.019931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020645 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.020876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.021939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.022956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.023889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.024970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.025889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026878 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.026996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027226 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.027906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.028960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.029930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.030981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031763 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.031987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.032956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.external = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.external = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.033930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.password = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.password = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] auth.token = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] auth.token = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.034967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.035888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.036961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.037880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.038919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.039039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.039153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.039274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.039388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.039992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040105 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.040939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.041962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042081 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.042887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.043994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.044977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.045896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.046926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.047958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048885 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.048998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.049920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.050938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.051938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.052907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.053797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.054935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.055047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.055760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.055877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.055999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.055999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.056212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.056212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.056437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.056550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.056673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.056788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.056901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.057922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.058953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.059974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.060936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061507 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.061958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.062996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063234 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.063918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.064876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065227 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.065911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.066883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.067915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.068969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.069999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.070991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071225 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.071908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072645 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.072982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.073898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074696 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.074920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.075951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.076990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.077988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.078910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079234 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.079922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080147 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.080889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.081920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.082941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 27 13:10:07.083061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.083061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 27 13:10:07.083061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 27 13:10:07.112719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.112949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.113177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.113261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.113412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.113641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.121469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.121950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.151231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.151400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.151485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.151627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.203767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.203919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.252548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.252663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.252796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.252875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.407846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.408953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.409079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.432299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.432486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.432571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.432719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.432810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.432939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.433785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.444514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.444738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.457759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.457895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.457979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.458988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.459068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.459195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 27 13:10:07.476267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.476484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 13:10:07.476629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.476831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.476911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 13:10:07.477061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 13:10:07.477196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.477341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.477418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 27 13:10:07.477606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.477684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.477916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.478123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.478389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.478467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.478679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.487654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.487887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.487968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.488168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.488279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.488465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.488556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.488750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.495429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.495650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.498340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.498524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.498632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.498811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.498920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.499096 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.499207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.499383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.499493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.499668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.499746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.499932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.500017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.500204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.500303 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.500502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.500683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.500889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.500970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.501166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.501244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.501437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.524827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.524985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.526286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.526442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.526574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.526728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.532894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.532987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.533185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.533265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.533471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.533567 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.539817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.539902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.540111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.540192 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.544236 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.544330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.544543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.544623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.544838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.544926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.545121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.545201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.545400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.545480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.545687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.545778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.545970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.546049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.546251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.546330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.561706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.561805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.561990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.562070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.565843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.565927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.569083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.569173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.569381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.569461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.574093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.574203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.577243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.577326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.580451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.580534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.580757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.580837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.581048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.581128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.581352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.581446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.581663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.581744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.581949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.582028 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.598155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.598311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.598449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.598608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.598720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.598880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.598975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.599146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.605816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.606034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.607340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.607574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.608915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.609158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.609239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.609451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.611805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.612079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.612190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.612395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.612506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.612779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.612898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.613064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.613171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.613350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.624110 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.624418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.624514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.624785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.624877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.625091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.625183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.625385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.625477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.625676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.625784 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.625960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.626052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.626287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.641943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.642406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.644847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.645330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.647711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.648200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.649793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.650281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.651392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.651686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.651880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.651961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.652185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.652500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.658276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.658804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.659735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.660298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.662714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.663003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.663266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.663358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.663553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.663642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.663839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.664122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.671336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.671628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.671910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.671992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.672200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.672509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.679677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.679963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.680298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.680381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 27 13:10:07.680579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.680671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.680856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.680944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.681128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.681212 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.681399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.681481 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.681673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.681764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.681952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.682042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.682222 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.682304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.682492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.682573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.682780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.682902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.683056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.683137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.683322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.683402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.683589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.683855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 27 13:10:07.717007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55839fe169c0 pid: 81441 (default app) Oct 27 13:10:07.717619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55839fe169c0 pid: 81442 (default app) Oct 27 13:10:07.720242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eaeb1-83a2-4c94-9429-92ce7c196f3d None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:07.720536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eaeb1-83a2-4c94-9429-92ce7c196f3d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:07.720809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eaeb1-83a2-4c94-9429-92ce7c196f3d None None] PATH_INFO: `/v3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:07.724929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Thu Oct 27 13:10:07 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:14.449036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf8bbd-1301-4a4e-939a-76f4d1603b4d None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:14.449562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf8bbd-1301-4a4e-939a-76f4d1603b4d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:14.449562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf8bbd-1301-4a4e-939a-76f4d1603b4d None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:14.453805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:14.458944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:14.459521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:14.459521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:14.531598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: INFO dbcounter [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] Registered counter for database keystone Oct 27 13:10:14.550904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG oslo_db.sqlalchemy.engines [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 27 13:10:14.557739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writer thread running {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 27 13:10:14.736486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG passlib.handlers.bcrypt [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81441) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 27 13:10:14.736785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG passlib.handlers.bcrypt [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81441) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 27 13:10:14.795016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:14.796652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:14.923292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 13:10:15.356506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf8bbd-1301-4a4e-939a-76f4d1603b4d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:15.425720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: INFO dbcounter [None req-ffcf8bbd-1301-4a4e-939a-76f4d1603b4d None None] Registered counter for database keystone Oct 27 13:10:15.447568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG oslo_db.sqlalchemy.engines [None req-ffcf8bbd-1301-4a4e-939a-76f4d1603b4d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81442) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 27 13:10:15.453759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writer thread running {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 27 13:10:15.569323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf8bbd-1301-4a4e-939a-76f4d1603b4d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:15.574368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55df61-b817-4297-915f-6bdc7b751dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:15.574974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55df61-b817-4297-915f-6bdc7b751dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:15.575248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55df61-b817-4297-915f-6bdc7b751dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:15.575517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55df61-b817-4297-915f-6bdc7b751dc6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:15.583574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55df61-b817-4297-915f-6bdc7b751dc6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:15.635689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0d55df61-b817-4297-915f-6bdc7b751dc6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 27 13:10:15.636027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55df61-b817-4297-915f-6bdc7b751dc6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:15.645986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Oct 27 13:10:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 13:10:15.651974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:15.668270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cf17b-877b-42ba-beca-3ef75ad07ef1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:15.672618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17305c46-300b-46cc-bd08-564a6429bee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:15.673720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305c46-300b-46cc-bd08-564a6429bee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:15.673996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305c46-300b-46cc-bd08-564a6429bee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:15.674271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305c46-300b-46cc-bd08-564a6429bee4 admin admin] PATH_INFO: `/v3/projects/f354da2936f649f78cdcb89056983d15` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:15.677992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17305c46-300b-46cc-bd08-564a6429bee4 admin admin] RBAC: Authorizing `identity:get_project(project_id=f354da2936f649f78cdcb89056983d15)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:15.729163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING keystone.common.rbac_enforcer.enforcer [None req-17305c46-300b-46cc-bd08-564a6429bee4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 27 13:10:15.729498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17305c46-300b-46cc-bd08-564a6429bee4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:15.733872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Oct 27 13:10:15 2022] GET /identity/v3/projects/f354da2936f649f78cdcb89056983d15 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:17.831191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bdf7b-1116-41e1-a7da-b530285b1cfb None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:17.831748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bdf7b-1116-41e1-a7da-b530285b1cfb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:17.831748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bdf7b-1116-41e1-a7da-b530285b1cfb None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:17.835768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:17.841328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990921a-b225-4f26-bd87-376b0aac2a57 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:17.841735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990921a-b225-4f26-bd87-376b0aac2a57 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:17.841872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990921a-b225-4f26-bd87-376b0aac2a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:17.861360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6990921a-b225-4f26-bd87-376b0aac2a57 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:17.862837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6990921a-b225-4f26-bd87-376b0aac2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:17.871095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 27 13:10:18.138685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bdf7b-1116-41e1-a7da-b530285b1cfb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:18.150806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bdf7b-1116-41e1-a7da-b530285b1cfb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:18.155018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2844cf-cafd-42bf-ab85-7bb48615c394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:18.155609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2844cf-cafd-42bf-ab85-7bb48615c394 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:18.155887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2844cf-cafd-42bf-ab85-7bb48615c394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:18.156158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2844cf-cafd-42bf-ab85-7bb48615c394 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:18.163648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2844cf-cafd-42bf-ab85-7bb48615c394 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:18.165975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2844cf-cafd-42bf-ab85-7bb48615c394 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:18.174477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Oct 27 13:10:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:10:18.179583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990921a-b225-4f26-bd87-376b0aac2a57 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:18.188988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990921a-b225-4f26-bd87-376b0aac2a57 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:18.193210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48f109-a2d8-4217-a1c7-a51732365feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:18.193855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f109-a2d8-4217-a1c7-a51732365feb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:18.194131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f109-a2d8-4217-a1c7-a51732365feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:18.194397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f109-a2d8-4217-a1c7-a51732365feb admin admin] PATH_INFO: `/v3/users/299712f6a5bb49b986da988b00e6dbe7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:18.195699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48f109-a2d8-4217-a1c7-a51732365feb admin admin] RBAC: Authorizing `identity:get_user(user_id=299712f6a5bb49b986da988b00e6dbe7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:18.197565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48f109-a2d8-4217-a1c7-a51732365feb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:18.199053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Thu Oct 27 13:10:18 2022] GET /identity/v3/users/299712f6a5bb49b986da988b00e6dbe7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:20.384399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64baf1-8bbe-4de1-bf22-20fbad7ce538 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:20.384913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64baf1-8bbe-4de1-bf22-20fbad7ce538 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:20.385012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64baf1-8bbe-4de1-bf22-20fbad7ce538 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:20.386259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:20.391867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f4e02-4a29-42a9-a5dd-04b3065e59a7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:20.392288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f4e02-4a29-42a9-a5dd-04b3065e59a7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:20.392458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f4e02-4a29-42a9-a5dd-04b3065e59a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:20.404400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a28aa-b04b-4683-987d-d6f34ef30d66 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:20.404688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a28aa-b04b-4683-987d-d6f34ef30d66 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:20.404963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a28aa-b04b-4683-987d-d6f34ef30d66 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:20.406271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:10:20.411191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f24f4e02-4a29-42a9-a5dd-04b3065e59a7 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:20.411323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:20.411585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:20.411854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:20.412388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f24f4e02-4a29-42a9-a5dd-04b3065e59a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:20.453205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG passlib.handlers.bcrypt [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81442) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 27 13:10:20.453700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG passlib.handlers.bcrypt [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81442) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 27 13:10:20.493430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:20.502405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:20.503972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:20.601892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:20.759214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f4e02-4a29-42a9-a5dd-04b3065e59a7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:20.783049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:20.863530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f4e02-4a29-42a9-a5dd-04b3065e59a7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:20.877046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:20.896811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:20.897449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:20.897726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:20.898002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:20.909466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:20.918791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:20.919671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:20.920810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:10:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:20.925554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:20.978616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684ae7c-9a2e-467f-b392-53bf58d2f924 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:20.981453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:21.004809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:21.017098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:21.017907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:21.018190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:21.018456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:21.019401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:21.020851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:21.023506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a318ec8-e8df-4ff6-a0f7-630e796c04bc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:21.032448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.sql.core [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 13:10:21.032448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 13:10:21.032448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [parameters: {'id': 'a5b3e0c043f94e37afcb70803e71746f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 13:10:21.032448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81442) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:10:21.050702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result.read() Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 27 13:10:21.051596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application yield resource Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise self.value Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application session.commit() Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 13:10:21.052778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.session.flush() Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application compat.raise_( Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 13:10:21.053835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 13:10:21.054679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result.read() Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:10:21.055405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [parameters: {'id': 'a5b3e0c043f94e37afcb70803e71746f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 27 13:10:21.056191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:21.057577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 8/15] 10.222.0.48 () {60 vars in 1137 bytes} [Thu Oct 27 13:10:20 2022] POST /identity/v3/domains => generated 190 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:10:21.059119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:21.062491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:21.063103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:21.063379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:21.063650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:21.064352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:21.064655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:21.066086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:21.078285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 8/16] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:21.083714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:21.156555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90672c81-7ecd-4a61-9f3a-78cb0e642522 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:21.179439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c7116-6efc-41a1-8ef2-893c5838fd81 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:21.189390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:21.190231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:21.190510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:21.190780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:21.208922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:21.210383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:21.212819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:21.213447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:21.213722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:21.213991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] PATH_INFO: `/v3/users/299712f6a5bb49b986da988b00e6dbe7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:21.215123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] RBAC: Authorizing `identity:get_user(user_id=299712f6a5bb49b986da988b00e6dbe7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:21.216604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:21.218188 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:21 2022] GET /identity/v3/users/299712f6a5bb49b986da988b00e6dbe7 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:21.223551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:10:21.230017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:21.231048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:10:21.232040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:10:21.232040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:10:21.232040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:21.232040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:10:21.238127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:21.238647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:21.260430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:21.326027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dbc1d-7b63-428a-8cc1-1fdf3f725730 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:21.336462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c6612-89c3-4e86-a55d-b1e405bfc12b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:21.359144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:21.359983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:21.360287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:21.360559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:21.361674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:21.363156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:21.364728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:21.370796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:21.370930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13f48c-d7b6-4faf-857b-37779bb986b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:21.371414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f48c-d7b6-4faf-857b-37779bb986b8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:21.371690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f48c-d7b6-4faf-857b-37779bb986b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:21.371955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f48c-d7b6-4faf-857b-37779bb986b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:21.372623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f48c-d7b6-4faf-857b-37779bb986b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:21.372921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f48c-d7b6-4faf-857b-37779bb986b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:21.374281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f48c-d7b6-4faf-857b-37779bb986b8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:21.392260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:21.455480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:21.475893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f5e57-e624-42d0-a88d-a03058509a80 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:21.512122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a5dbc-00b6-41be-9da7-0d5012c0613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:21.512829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5dbc-00b6-41be-9da7-0d5012c0613d None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:21.513103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5dbc-00b6-41be-9da7-0d5012c0613d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:21.513375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5dbc-00b6-41be-9da7-0d5012c0613d None admin] PATH_INFO: `/v3/domains/default/users/299712f6a5bb49b986da988b00e6dbe7/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:21.515646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a5dbc-00b6-41be-9da7-0d5012c0613d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=299712f6a5bb49b986da988b00e6dbe7, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:21.517134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a5dbc-00b6-41be-9da7-0d5012c0613d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:21.630665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 11/21] 10.222.0.48 () {58 vars in 1361 bytes} [Thu Oct 27 13:10:21 2022] PUT /identity/v3/domains/default/users/299712f6a5bb49b986da988b00e6dbe7/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:10:23.992048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029978a-4eff-4592-80cf-4b2b9f556563 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:23.993175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029978a-4eff-4592-80cf-4b2b9f556563 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:23.993693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029978a-4eff-4592-80cf-4b2b9f556563 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:23.995297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:24.001552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b6c31-4e8a-4b38-abda-64e9fc930394 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.002375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b6c31-4e8a-4b38-abda-64e9fc930394 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.002879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b6c31-4e8a-4b38-abda-64e9fc930394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.031294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ab2b6c31-4e8a-4b38-abda-64e9fc930394 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:24.033157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ab2b6c31-4e8a-4b38-abda-64e9fc930394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:24.044521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cf9a3-98a9-4ba0-900a-ce6533ca0dc0 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.045116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cf9a3-98a9-4ba0-900a-ce6533ca0dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.045649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cf9a3-98a9-4ba0-900a-ce6533ca0dc0 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.047563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 12/23] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:24.055018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad516d1-e516-44c1-baa5-68d4d9e81fb2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.055820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad516d1-e516-44c1-baa5-68d4d9e81fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.056700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad516d1-e516-44c1-baa5-68d4d9e81fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.090034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5ad516d1-e516-44c1-baa5-68d4d9e81fb2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:24.091740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5ad516d1-e516-44c1-baa5-68d4d9e81fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:24.181911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 12/24] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:24.193215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983500cd-03e0-4d5b-a36f-6679d1b2706b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.193508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983500cd-03e0-4d5b-a36f-6679d1b2706b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.193788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983500cd-03e0-4d5b-a36f-6679d1b2706b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.198694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 13/25] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:24.204803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baad5e-c80a-45ff-a74e-b9173ab91910 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.205091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baad5e-c80a-45ff-a74e-b9173ab91910 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.205371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baad5e-c80a-45ff-a74e-b9173ab91910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.215297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 13/26] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:24.237304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-28baad5e-c80a-45ff-a74e-b9173ab91910 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:24.238519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-28baad5e-c80a-45ff-a74e-b9173ab91910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:24.318421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:24.325306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 14/27] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:24.645683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad516d1-e516-44c1-baa5-68d4d9e81fb2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:24.658719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:24.674634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baad5e-c80a-45ff-a74e-b9173ab91910 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:24.803242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baad5e-c80a-45ff-a74e-b9173ab91910 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:24.815942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad516d1-e516-44c1-baa5-68d4d9e81fb2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:24.872582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:24.873150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.873433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.873704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.892881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:24.894486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:24.897858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889e5d2-54a0-4c67-bc7e-f284c2d83591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:24.898502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e5d2-54a0-4c67-bc7e-f284c2d83591 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.898778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e5d2-54a0-4c67-bc7e-f284c2d83591 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.899046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889e5d2-54a0-4c67-bc7e-f284c2d83591 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.918686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889e5d2-54a0-4c67-bc7e-f284c2d83591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:10:24.919949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:24.920832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:10:24.921630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:10:24.921630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:10:24.921630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:24.921630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 15/28] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:10:24.922285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3889e5d2-54a0-4c67-bc7e-f284c2d83591 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:24.925488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:24.926394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:24.927146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 14/29] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:10:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:24.928086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c8172-cb00-401f-815b-00595479b79e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:24.928538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c8172-cb00-401f-815b-00595479b79e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:24.928941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c8172-cb00-401f-815b-00595479b79e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:24.932691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 15/30] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:24.933551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c8172-cb00-401f-815b-00595479b79e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:24.998261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:25.027715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:25.059363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfbb9a-cd35-410b-9ded-468805110c82 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.062977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c8172-cb00-401f-815b-00595479b79e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.097225 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:25.097426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5731ffd-a47a-4253-98f3-cded6fa57768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:25.097835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.098111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.098381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.098637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5731ffd-a47a-4253-98f3-cded6fa57768 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.099023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:25.099252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5731ffd-a47a-4253-98f3-cded6fa57768 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.099353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:25.099752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5731ffd-a47a-4253-98f3-cded6fa57768 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.100607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5731ffd-a47a-4253-98f3-cded6fa57768 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:25.100789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:25.102368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5731ffd-a47a-4253-98f3-cded6fa57768 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:25.114406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 16/31] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:25.117197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:25.136474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 16/32] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Oct 27 13:10:24 2022] POST /identity/v3/roles => generated 219 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:25.138263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a57357-261e-43cf-ab3b-e5fc706b32c0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.138720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a57357-261e-43cf-ab3b-e5fc706b32c0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.139133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a57357-261e-43cf-ab3b-e5fc706b32c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.172378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-18a57357-261e-43cf-ab3b-e5fc706b32c0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:25.173896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-18a57357-261e-43cf-ab3b-e5fc706b32c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:25.232612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d571fd2-66ee-4a46-88a2-c39b2bdeae82 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.266200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:25.273560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:25.274227 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.275937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.276246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.276957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:25.277259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:25.279662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:25.307165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 17/33] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:25.308021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:25.319469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 17/34] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:25.320361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a57357-261e-43cf-ab3b-e5fc706b32c0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:25.412621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:25.473742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe93de7-4f9f-421c-99c0-6f2e34660903 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.502108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a57357-261e-43cf-ab3b-e5fc706b32c0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.512669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadef82-a277-4ca5-aabd-b7d9f190512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:25.513168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadef82-a277-4ca5-aabd-b7d9f190512d None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.513446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadef82-a277-4ca5-aabd-b7d9f190512d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.513716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadef82-a277-4ca5-aabd-b7d9f190512d None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.514367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadef82-a277-4ca5-aabd-b7d9f190512d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:25.516275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadef82-a277-4ca5-aabd-b7d9f190512d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:25.550319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:25.551332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.551926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.552421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] PATH_INFO: `/v3/users/299712f6a5bb49b986da988b00e6dbe7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.553878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] RBAC: Authorizing `identity:get_user(user_id=299712f6a5bb49b986da988b00e6dbe7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:25.555569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:25.557476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 18/35] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:25 2022] GET /identity/v3/users/299712f6a5bb49b986da988b00e6dbe7 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:25.564987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:25.694859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 18/36] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:10:25 2022] POST /identity/v3/projects => generated 301 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:25.698750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144204-dca1-4623-9eee-b7d74245f2cb None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.698750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144204-dca1-4623-9eee-b7d74245f2cb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.704495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144204-dca1-4623-9eee-b7d74245f2cb None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.706664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 19/37] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:25.713856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2c5fd-7b6c-45c1-ab1f-45aa997fe027 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.714480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2c5fd-7b6c-45c1-ab1f-45aa997fe027 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.715329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2c5fd-7b6c-45c1-ab1f-45aa997fe027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.720709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44279b-fd8a-4514-8251-eb19bca46a3c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.746285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:25.756284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-45f2c5fd-7b6c-45c1-ab1f-45aa997fe027 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:25.761001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-45f2c5fd-7b6c-45c1-ab1f-45aa997fe027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:25.795436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:25.796821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.797345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.798387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.800484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:25.803036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:25.806113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 19/38] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:25.807270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:25.818501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:25.850716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 20/39] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:25.851712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2c5fd-7b6c-45c1-ab1f-45aa997fe027 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:25.932913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b10d-fff0-49b9-a5e2-ccd3389ba1f7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.972865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2c5fd-7b6c-45c1-ab1f-45aa997fe027 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:25.973562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a6501-18d7-408d-bc97-cace38703054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:25.974386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a6501-18d7-408d-bc97-cace38703054 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.974777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a6501-18d7-408d-bc97-cace38703054 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.975174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a6501-18d7-408d-bc97-cace38703054 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.975950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a6501-18d7-408d-bc97-cace38703054 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:25.977570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a6501-18d7-408d-bc97-cace38703054 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:25.992549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 20/40] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Oct 27 13:10:25 2022] POST /identity/v3/roles => generated 225 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:25.994618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac09478-cc04-43b8-a1ac-22ae303f0464 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:25.994906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac09478-cc04-43b8-a1ac-22ae303f0464 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:25.995177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac09478-cc04-43b8-a1ac-22ae303f0464 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:25.996826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 21/41] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:26.005555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdeb2-27fb-4a81-88a2-5e79ae63898e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:26.005850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdeb2-27fb-4a81-88a2-5e79ae63898e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:26.006137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdeb2-27fb-4a81-88a2-5e79ae63898e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:26.022569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:26.023574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:26.024000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:26.024677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:26.025689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.domain.id=default, user.id=299712f6a5bb49b986da988b00e6dbe7) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:26.026179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:26.028168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:26.039572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 21/42] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Oct 27 13:10:25 2022] GET /identity/v3/role_assignments?user.id=299712f6a5bb49b986da988b00e6dbe7&scope.domain.id=default&role.id=33f912ad05b44e36a696ef6aa86733be => generated 530 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:26.138367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:26.144859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a9bcdeb2-27fb-4a81-88a2-5e79ae63898e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:26.146159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a9bcdeb2-27fb-4a81-88a2-5e79ae63898e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:26.183602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:26.203404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:26.234730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 22/43] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:26.296902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957f185-768f-41a2-b1b2-5265e6d8a8ad None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:26.331433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:26.332047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:26.332342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:26.332609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:26.333273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:26.334650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:26.350956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 22/44] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Oct 27 13:10:26 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:10:26.510199 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcdeb2-27fb-4a81-88a2-5e79ae63898e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:26.607372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcdeb2-27fb-4a81-88a2-5e79ae63898e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:26.641114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f6efc-7187-4ee2-94c4-e1f12dbfe269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:26.641745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f6efc-7187-4ee2-94c4-e1f12dbfe269 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:26.642025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f6efc-7187-4ee2-94c4-e1f12dbfe269 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:26.642296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f6efc-7187-4ee2-94c4-e1f12dbfe269 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:26.643390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f6efc-7187-4ee2-94c4-e1f12dbfe269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:26.644871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f6efc-7187-4ee2-94c4-e1f12dbfe269 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:26.646416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 23/45] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:26.652085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:26.742358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:26.746296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a482e-e0f7-4297-852f-87883cac992e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:26.779735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d8d7d-36f0-44a8-91f9-3c8cd831810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:26.780297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d8d7d-36f0-44a8-91f9-3c8cd831810d None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:26.780580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d8d7d-36f0-44a8-91f9-3c8cd831810d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:26.780853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d8d7d-36f0-44a8-91f9-3c8cd831810d None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:26.781502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d8d7d-36f0-44a8-91f9-3c8cd831810d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:26.782934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d8d7d-36f0-44a8-91f9-3c8cd831810d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:26.807276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 23/46] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:10:26 2022] POST /identity/v3/projects => generated 312 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:28.939541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d66f8-c9bf-4cc3-a3e1-85798ffd3cc7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:28.940042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d66f8-c9bf-4cc3-a3e1-85798ffd3cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:28.940042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d66f8-c9bf-4cc3-a3e1-85798ffd3cc7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:28.941374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 24/47] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:28.946699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1ed8-4d6f-48f9-a9d7-c222201defb3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:28.947101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1ed8-4d6f-48f9-a9d7-c222201defb3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:28.947251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1ed8-4d6f-48f9-a9d7-c222201defb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:28.965528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-eb1e1ed8-4d6f-48f9-a9d7-c222201defb3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:28.966699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-eb1e1ed8-4d6f-48f9-a9d7-c222201defb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:29.030854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 24/48] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:29.299642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d66f8-c9bf-4cc3-a3e1-85798ffd3cc7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:29.334432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:29.396449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d66f8-c9bf-4cc3-a3e1-85798ffd3cc7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:29.428552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0113e-30e1-4fd1-8b55-2b868c07cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:29.429164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0113e-30e1-4fd1-8b55-2b868c07cae5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:29.429437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0113e-30e1-4fd1-8b55-2b868c07cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:29.429702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0113e-30e1-4fd1-8b55-2b868c07cae5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:29.431375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0113e-30e1-4fd1-8b55-2b868c07cae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:29.432832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0113e-30e1-4fd1-8b55-2b868c07cae5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:29.434350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 25/49] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:29.439780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e1ed8-4d6f-48f9-a9d7-c222201defb3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:29.620441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:29.639433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e1ed8-4d6f-48f9-a9d7-c222201defb3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:29.672399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562de5c8-7987-4752-a214-170b75fcf2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:29.673009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562de5c8-7987-4752-a214-170b75fcf2fb None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:29.673287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562de5c8-7987-4752-a214-170b75fcf2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:29.673556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562de5c8-7987-4752-a214-170b75fcf2fb None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:29.674193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562de5c8-7987-4752-a214-170b75fcf2fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:29.675640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562de5c8-7987-4752-a214-170b75fcf2fb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:29.699801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 25/50] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Oct 27 13:10:29 2022] POST /identity/v3/projects => generated 298 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:31.816894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d819b1-37bb-470d-9ae8-37299176677b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:31.817391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d819b1-37bb-470d-9ae8-37299176677b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:31.817503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d819b1-37bb-470d-9ae8-37299176677b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:31.818775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 26/51] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:31.824372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508d34-b702-4f19-ae10-dfc04e019e25 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:31.824734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508d34-b702-4f19-ae10-dfc04e019e25 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:31.824918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508d34-b702-4f19-ae10-dfc04e019e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:31.842892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-aa508d34-b702-4f19-ae10-dfc04e019e25 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:31.844057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aa508d34-b702-4f19-ae10-dfc04e019e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:31.907187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 26/52] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:32.172460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d819b1-37bb-470d-9ae8-37299176677b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:32.271987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d819b1-37bb-470d-9ae8-37299176677b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:32.274223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:32.304598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6060ad-376e-4b81-aae4-5e96863541b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:32.305214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6060ad-376e-4b81-aae4-5e96863541b1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:32.305490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6060ad-376e-4b81-aae4-5e96863541b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:32.306394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6060ad-376e-4b81-aae4-5e96863541b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:32.307490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6060ad-376e-4b81-aae4-5e96863541b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:32.308944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6060ad-376e-4b81-aae4-5e96863541b1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:32.310484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 27/53] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:32.315955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508d34-b702-4f19-ae10-dfc04e019e25 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:32.373420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:32.410234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508d34-b702-4f19-ae10-dfc04e019e25 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:32.442578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4da543-6b7c-43bc-abec-0b26f956bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:32.443123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da543-6b7c-43bc-abec-0b26f956bc58 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:32.443401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da543-6b7c-43bc-abec-0b26f956bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:32.443669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da543-6b7c-43bc-abec-0b26f956bc58 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:32.444322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da543-6b7c-43bc-abec-0b26f956bc58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:32.445744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da543-6b7c-43bc-abec-0b26f956bc58 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:32.470642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 27/54] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:10:32 2022] POST /identity/v3/users => generated 275 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:35.747679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4061e-29f8-471c-ac28-dc92bb92e053 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:35.748643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4061e-29f8-471c-ac28-dc92bb92e053 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:35.749148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4061e-29f8-471c-ac28-dc92bb92e053 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:35.750699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 28/55] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:35.755978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed926f37-97e4-4906-b837-f6b7ab970d3f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:35.756393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed926f37-97e4-4906-b837-f6b7ab970d3f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:35.756591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed926f37-97e4-4906-b837-f6b7ab970d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:35.781459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ed926f37-97e4-4906-b837-f6b7ab970d3f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:35.782627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ed926f37-97e4-4906-b837-f6b7ab970d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:35.787229 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ac9af-bca4-4f2d-884a-31910b635fad None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:35.787738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ac9af-bca4-4f2d-884a-31910b635fad None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:35.788252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ac9af-bca4-4f2d-884a-31910b635fad None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:35.790087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 29/56] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:35.805041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5e4df-4dd7-4100-a2f5-f1b2cb805fdc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:35.805584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5e4df-4dd7-4100-a2f5-f1b2cb805fdc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:35.806439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5e4df-4dd7-4100-a2f5-f1b2cb805fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:35.841284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4ec5e4df-4dd7-4100-a2f5-f1b2cb805fdc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:35.842480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4ec5e4df-4dd7-4100-a2f5-f1b2cb805fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:35.889091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 28/57] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:35.934716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a81762-0da3-44a3-92e3-68d0b1510680 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:35.935019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a81762-0da3-44a3-92e3-68d0b1510680 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:35.935284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a81762-0da3-44a3-92e3-68d0b1510680 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:35.936721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 29/58] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:10:35.950606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7ff4b-717f-4636-9607-9d4cb7650a34 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:35.950912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7ff4b-717f-4636-9607-9d4cb7650a34 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:35.951148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7ff4b-717f-4636-9607-9d4cb7650a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:35.974994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b3c7ff4b-717f-4636-9607-9d4cb7650a34 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:35.976184 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b3c7ff4b-717f-4636-9607-9d4cb7650a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:35.982124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 30/59] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:36.041000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:36.052849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde4843-9494-4d3e-9eff-e34145f85707 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.053130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde4843-9494-4d3e-9eff-e34145f85707 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.053405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde4843-9494-4d3e-9eff-e34145f85707 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.054752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 31/60] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:10:36.066397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c18b4-9fdd-41d8-a80c-c1a6ba872557 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.066673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c18b4-9fdd-41d8-a80c-c1a6ba872557 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.066944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c18b4-9fdd-41d8-a80c-c1a6ba872557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.085027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 30/61] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:36.089152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b33c18b4-9fdd-41d8-a80c-c1a6ba872557 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:36.090312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b33c18b4-9fdd-41d8-a80c-c1a6ba872557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:36.180062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 32/62] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:36.291215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7ff4b-717f-4636-9607-9d4cb7650a34 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.319118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c18b4-9fdd-41d8-a80c-c1a6ba872557 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.364209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:36.418535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7ff4b-717f-4636-9607-9d4cb7650a34 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.444896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c18b4-9fdd-41d8-a80c-c1a6ba872557 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.458692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.459532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.459946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.460381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.473050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:36.480340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:36.481241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:36.481975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:36.484319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 31/63] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:36.485298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.487312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.488071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.488479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.489153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.502911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:36.509407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:36.510247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:36.511070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 33/64] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:10:36.511666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.538814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:36.610883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816aac1-7ea9-48a5-b3ed-5a6b9fdc8d6e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.619401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:36.629515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ad4f9-e776-4d91-8e05-6e39e627b5e0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.648204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.648863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.649141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.649409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.660189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:36.666515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:36.667343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:36.668073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 32/65] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:36.668734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.668734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.668734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.668734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.669508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:36.669597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:36.670928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:36.672219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.680095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 34/66] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:36.680991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.770802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:36.774355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95fd9-5027-431e-a9ea-cb8ed96f843e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.779985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c78699-5411-484a-88b1-8102c9e938df None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.808033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.808671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.808948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.809219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.809869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:36.810166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:36.811539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:36.812668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.813300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.813575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.813849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.819511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 33/67] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:36.820309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.824301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:36.829714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:36.830715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:36.831469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 35/68] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:36.832253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.847432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:36.921449 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e02aa-1e06-4521-bb37-39d6606284ec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.928689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e40f0e-1611-42a8-9be6-61e6a553dc2f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:36.953519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fb00c-1c04-4e28-94f9-cfa159cb9ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.954210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fb00c-1c04-4e28-94f9-cfa159cb9ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.954497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fb00c-1c04-4e28-94f9-cfa159cb9ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.954763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fb00c-1c04-4e28-94f9-cfa159cb9ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.955403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fb00c-1c04-4e28-94f9-cfa159cb9ac8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:36.955705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fb00c-1c04-4e28-94f9-cfa159cb9ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:36.957108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fb00c-1c04-4e28-94f9-cfa159cb9ac8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:36.961168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:36.961799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.962643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.962931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] PATH_INFO: `/v3/users/b3d5364c4bfc467ba54c1022c639c48c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.966024 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 34/69] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:36.966862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cc01a-59d5-477b-ad67-13c5db60f46b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:36.967146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cc01a-59d5-477b-ad67-13c5db60f46b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:36.967413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cc01a-59d5-477b-ad67-13c5db60f46b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:36.972373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 35/70] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:36.973366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cc01a-59d5-477b-ad67-13c5db60f46b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.976075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d5364c4bfc467ba54c1022c639c48c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:36.977566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:36.979337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 36/71] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/users/b3d5364c4bfc467ba54c1022c639c48c => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:36.980086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:36.997201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.050188 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.073996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cc01a-59d5-477b-ad67-13c5db60f46b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.078939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e810-d1f1-4b4c-a983-4bf31b4e1ff2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.107689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.108345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.108624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.108894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] PATH_INFO: `/v3/users/299712f6a5bb49b986da988b00e6dbe7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.110237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] RBAC: Authorizing `identity:get_user(user_id=299712f6a5bb49b986da988b00e6dbe7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.111682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.112710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7552d11-9f7c-41ce-828f-8d5fdd135d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.113289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 36/72] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/users/299712f6a5bb49b986da988b00e6dbe7 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.117861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7552d11-9f7c-41ce-828f-8d5fdd135d9e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.117861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7552d11-9f7c-41ce-828f-8d5fdd135d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.117861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7552d11-9f7c-41ce-828f-8d5fdd135d9e None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.118053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.118120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7552d11-9f7c-41ce-828f-8d5fdd135d9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:37.118120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7552d11-9f7c-41ce-828f-8d5fdd135d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.118120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7552d11-9f7c-41ce-828f-8d5fdd135d9e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.125885 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 37/73] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.127043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e8b0-a366-4849-b15d-a8ed2d2c5a2b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.127327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e8b0-a366-4849-b15d-a8ed2d2c5a2b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.127597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e8b0-a366-4849-b15d-a8ed2d2c5a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.145211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d502e8b0-a366-4849-b15d-a8ed2d2c5a2b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:37.146380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d502e8b0-a366-4849-b15d-a8ed2d2c5a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:37.206084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.210593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 38/74] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:37.211400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e8b0-a366-4849-b15d-a8ed2d2c5a2b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.216861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c503b34-b252-4078-ac77-f91b44856f9d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.253327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.253961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.254239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.254511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] PATH_INFO: `/v3/users/b3d5364c4bfc467ba54c1022c639c48c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.255636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d5364c4bfc467ba54c1022c639c48c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.257083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.258798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 37/75] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:37 2022] GET /identity/v3/users/b3d5364c4bfc467ba54c1022c639c48c => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.259960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.281964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.321680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e8b0-a366-4849-b15d-a8ed2d2c5a2b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.356697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.357324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.357599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.357866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.359028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.360532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.362280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 39/76] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:37 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.362851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.371900 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf251368-45a0-4eb2-a391-e56b25610869 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.407230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.407859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.408136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.408459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.410144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.411572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.418459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 38/77] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:37 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.418459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.442343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.473456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7b6c7-3c97-4725-9cea-53bcc57d0d10 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.502492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.508064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.508727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.509526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.509994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] PATH_INFO: `/v3/users/b3d5364c4bfc467ba54c1022c639c48c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.510918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] RBAC: Authorizing `identity:get_user(user_id=b3d5364c4bfc467ba54c1022c639c48c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.512367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.513931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 40/78] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:37 2022] GET /identity/v3/users/b3d5364c4bfc467ba54c1022c639c48c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.517171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.517245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d26c2-7a87-458d-ad51-c76a9f8bf5cb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.550781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.551409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.551690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.551962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.553105 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.554547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.556196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 39/79] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:37 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.556953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.617388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157ba43-e33a-4194-83f1-45ae33293f7a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.650765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.651194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.651854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.652130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.652421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/b3d5364c4bfc467ba54c1022c639c48c/roles/fc711bb446524efbbb3ee062f6ff2702` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.654612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcc09680cf024f28a65aaca40b00c00f, user_id=b3d5364c4bfc467ba54c1022c639c48c, role_id=fc711bb446524efbbb3ee062f6ff2702)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.656055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.658250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342d6d2-a58c-423c-9b1e-fde38bb084b3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.668074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 41/80] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:37 2022] PUT /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/b3d5364c4bfc467ba54c1022c639c48c/roles/fc711bb446524efbbb3ee062f6ff2702 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:10:37.669040 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.695246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.695934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.696214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.697154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/299712f6a5bb49b986da988b00e6dbe7/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.699192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] RBAC: Authorizing `identity:create_grant(project_id=dcc09680cf024f28a65aaca40b00c00f, user_id=299712f6a5bb49b986da988b00e6dbe7, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.700712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.718674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 40/81] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:37 2022] PUT /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/299712f6a5bb49b986da988b00e6dbe7/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:10:37.719953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.772541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:37.829973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889990bc-5972-4b3f-9910-7aaf66718975 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.864277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2350ee-2746-4840-9b98-312bcb1a3a3c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:37.893133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.893735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.894017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.894288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.896649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.898849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.905704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 42/82] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.905704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.912048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:37.912817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:37.913102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:37.913377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] PATH_INFO: `/v3/projects/922bd7e9333c4789bb68ed75fa5cef9e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:37.914507 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] RBAC: Authorizing `identity:get_project(project_id=922bd7e9333c4789bb68ed75fa5cef9e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:37.916030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:37.917814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 41/83] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:37 2022] GET /identity/v3/projects/922bd7e9333c4789bb68ed75fa5cef9e => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:37.918554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:37.949404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:38.022747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120affc0-a036-44e9-8458-ae0e03ec0a1a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:38.054239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285538a-d110-4d1e-8e05-67c758520ceb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:38.067053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:38.078973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:38.080187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:38.081118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:38.081776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/b3d5364c4bfc467ba54c1022c639c48c/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:38.084727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcc09680cf024f28a65aaca40b00c00f, user_id=b3d5364c4bfc467ba54c1022c639c48c, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:38.086895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:38.092432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e15a-7081-4763-bc08-f63732fe8e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:38.092969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e15a-7081-4763-bc08-f63732fe8e3a None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:38.093313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e15a-7081-4763-bc08-f63732fe8e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:38.093658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e15a-7081-4763-bc08-f63732fe8e3a None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:38.094400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e15a-7081-4763-bc08-f63732fe8e3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:38.095882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e15a-7081-4763-bc08-f63732fe8e3a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:38.103852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 43/84] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:37 2022] PUT /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/b3d5364c4bfc467ba54c1022c639c48c/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:10:38.105389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:38.137730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 42/85] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:10:37 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:38.232139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381834c5-7029-4138-a1ea-076c82d04bc9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:38.275314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349af46a-1068-4693-b044-fe20bb97c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:38.276350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349af46a-1068-4693-b044-fe20bb97c9d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:38.276814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349af46a-1068-4693-b044-fe20bb97c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:38.277307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349af46a-1068-4693-b044-fe20bb97c9d2 None admin] PATH_INFO: `/v3/projects/922bd7e9333c4789bb68ed75fa5cef9e/users/b3d5364c4bfc467ba54c1022c639c48c/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:38.279626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349af46a-1068-4693-b044-fe20bb97c9d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=922bd7e9333c4789bb68ed75fa5cef9e, user_id=b3d5364c4bfc467ba54c1022c639c48c, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:38.281362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349af46a-1068-4693-b044-fe20bb97c9d2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:38.299612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 44/86] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:38 2022] PUT /identity/v3/projects/922bd7e9333c4789bb68ed75fa5cef9e/users/b3d5364c4bfc467ba54c1022c639c48c/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:10:40.483879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0026-d433-4aee-9057-4a98f39c506e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:40.484420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0026-d433-4aee-9057-4a98f39c506e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:40.484420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0026-d433-4aee-9057-4a98f39c506e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:40.485924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 43/87] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:40.501846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565ec95-3bd7-4d5b-a772-2baefbec88f2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:40.502339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565ec95-3bd7-4d5b-a772-2baefbec88f2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:40.502412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565ec95-3bd7-4d5b-a772-2baefbec88f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:40.529214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5565ec95-3bd7-4d5b-a772-2baefbec88f2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:40.530393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5565ec95-3bd7-4d5b-a772-2baefbec88f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:40.535182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e4b03-37ee-4663-8d84-343eace92dd7 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:40.535463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e4b03-37ee-4663-8d84-343eace92dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:40.535732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e4b03-37ee-4663-8d84-343eace92dd7 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:40.537273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 44/88] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:40.546790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad15973-6154-401b-9f07-65ea3c506e3c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:40.547070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad15973-6154-401b-9f07-65ea3c506e3c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:40.547347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad15973-6154-401b-9f07-65ea3c506e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:40.579892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6ad15973-6154-401b-9f07-65ea3c506e3c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:40.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ad15973-6154-401b-9f07-65ea3c506e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:40.661805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:40.717567 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 45/89] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:40.757831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 45/90] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:41.032955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554561f-0169-4d61-b90f-0d630a32b136 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.033138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554561f-0169-4d61-b90f-0d630a32b136 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.033417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554561f-0169-4d61-b90f-0d630a32b136 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.034771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 46/91] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:41.039876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d0e7-ad5d-410a-a283-efff95ced7ad None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.040166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d0e7-ad5d-410a-a283-efff95ced7ad None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.041035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d0e7-ad5d-410a-a283-efff95ced7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.065271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1911d0e7-ad5d-410a-a283-efff95ced7ad None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:41.066465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1911d0e7-ad5d-410a-a283-efff95ced7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:41.068568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:41.138938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554561f-0169-4d61-b90f-0d630a32b136 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:41.148767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 46/92] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:41.233417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911d0e7-ad5d-410a-a283-efff95ced7ad None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:41.300301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554561f-0169-4d61-b90f-0d630a32b136 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:41.356132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:41.357184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.357672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.358127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.379941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:41.390891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:41.391765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:41.392566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:41.398077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 47/93] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:10:41.405124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:41.444571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911d0e7-ad5d-410a-a283-efff95ced7ad None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:41.483819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b8e3c-58be-4545-8571-0420ffd5e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:41.484839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b8e3c-58be-4545-8571-0420ffd5e0ee None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.485738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b8e3c-58be-4545-8571-0420ffd5e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.486228 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:41.487347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b8e3c-58be-4545-8571-0420ffd5e0ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.496753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:41.501266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b8e3c-58be-4545-8571-0420ffd5e0ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-f46b8e3c-58be-4545-8571-0420ffd5e0ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:41.506620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:41.507442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:41.508194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:41.511269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 47/94] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:10:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:41.513336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d013a4-6bbc-4ebe-b31e-96949ef1a78f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.513816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d013a4-6bbc-4ebe-b31e-96949ef1a78f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.514278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d013a4-6bbc-4ebe-b31e-96949ef1a78f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.515958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 48/95] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:41.522166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7009a-d415-43cd-bfa6-5d1f0ec5b4ed None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.522752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7009a-d415-43cd-bfa6-5d1f0ec5b4ed None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.523261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7009a-d415-43cd-bfa6-5d1f0ec5b4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.542409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e3a7009a-d415-43cd-bfa6-5d1f0ec5b4ed None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:41.543871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e3a7009a-d415-43cd-bfa6-5d1f0ec5b4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:41.609100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cade1-0df7-477d-a6c7-b7ee35fa3de4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:41.624306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 49/96] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:41.625417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7009a-d415-43cd-bfa6-5d1f0ec5b4ed None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:41.651387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:41.652658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.653099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.653507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.654305 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:41.654738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:41.656806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:41.665933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 48/97] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:41.667320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:41.783521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:41.788002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952241a-517e-4d25-bad0-f90544ed6179 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:41.829268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:41.830363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.830885 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.831349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.846452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:41.847441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7009a-d415-43cd-bfa6-5d1f0ec5b4ed None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:41.853085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:41.854001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:41.854825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 49/98] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:41.855426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:41.886866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:41.896886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8a3e-593e-4b35-8cb3-990aaf0a41f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:41.897701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8a3e-593e-4b35-8cb3-990aaf0a41f5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.898168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8a3e-593e-4b35-8cb3-990aaf0a41f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.898575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8a3e-593e-4b35-8cb3-990aaf0a41f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.899503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f8a3e-593e-4b35-8cb3-990aaf0a41f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:41.899975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f8a3e-593e-4b35-8cb3-990aaf0a41f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:41.901676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f8a3e-593e-4b35-8cb3-990aaf0a41f5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:41.910520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 50/99] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:10:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:41.911931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e895a-0db9-447c-bee2-12ec53a345b9 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:41.912466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e895a-0db9-447c-bee2-12ec53a345b9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:41.912932 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e895a-0db9-447c-bee2-12ec53a345b9 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:41.914358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 51/100] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:41.915245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e895a-0db9-447c-bee2-12ec53a345b9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:41.979361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b830-26e9-4d58-b8ad-71bdc92dde3a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.013321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.013950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.014228 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.014501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] PATH_INFO: `/v3/users/299712f6a5bb49b986da988b00e6dbe7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.015880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] RBAC: Authorizing `identity:get_user(user_id=299712f6a5bb49b986da988b00e6dbe7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.017340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.018883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e895a-0db9-447c-bee2-12ec53a345b9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.019027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 50/101] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:41 2022] GET /identity/v3/users/299712f6a5bb49b986da988b00e6dbe7 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:42.019632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.046922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.054370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409d26c-59ee-4bad-8e42-6119348d258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.055055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409d26c-59ee-4bad-8e42-6119348d258b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.055342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409d26c-59ee-4bad-8e42-6119348d258b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.055617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409d26c-59ee-4bad-8e42-6119348d258b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.056867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409d26c-59ee-4bad-8e42-6119348d258b None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:42.057170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409d26c-59ee-4bad-8e42-6119348d258b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.058530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409d26c-59ee-4bad-8e42-6119348d258b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.066529 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 52/102] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:42.067709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924191-484f-414f-a889-47c8242fc2fe None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.068157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924191-484f-414f-a889-47c8242fc2fe None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.068471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924191-484f-414f-a889-47c8242fc2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.086221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ab924191-484f-414f-a889-47c8242fc2fe None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:42.087425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ab924191-484f-414f-a889-47c8242fc2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:42.115987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f42e5-423b-4e36-a6ad-b62f53bd3dcb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.132266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.148590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.149153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.149427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.150273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] PATH_INFO: `/v3/users/b3d5364c4bfc467ba54c1022c639c48c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.151383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d5364c4bfc467ba54c1022c639c48c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.152872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.153633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 53/103] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:42.154422 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 51/104] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/users/b3d5364c4bfc467ba54c1022c639c48c => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:42.155109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507ae92-d243-4ccd-9d56-f928957db76f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.155208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.155397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507ae92-d243-4ccd-9d56-f928957db76f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.155671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507ae92-d243-4ccd-9d56-f928957db76f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.160438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 54/105] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:42.160438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507ae92-d243-4ccd-9d56-f928957db76f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.251410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.258364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634b9e2-ea79-4ec1-8da3-20f63284ebc5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.265259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507ae92-d243-4ccd-9d56-f928957db76f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.297531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.298170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.298451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.298728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.300945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad691d8f-76e3-43c0-8321-ac76695aef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.301712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad691d8f-76e3-43c0-8321-ac76695aef11 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.302153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad691d8f-76e3-43c0-8321-ac76695aef11 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.302568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad691d8f-76e3-43c0-8321-ac76695aef11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.309964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:42.316369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:42.317408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 52/106] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:42.318345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.365781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.388454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad691d8f-76e3-43c0-8321-ac76695aef11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.390125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad691d8f-76e3-43c0-8321-ac76695aef11 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.427665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9ec1a-2347-4b30-8e01-2a57543a423f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.460478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.461109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.461393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.461684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] PATH_INFO: `/v3/users/b3d5364c4bfc467ba54c1022c639c48c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.462828 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d5364c4bfc467ba54c1022c639c48c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.464311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.466227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 53/107] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/users/b3d5364c4bfc467ba54c1022c639c48c => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:42.466999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.491351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.491524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 55/108] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:42.492645 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5d21-18ca-4f12-8e2f-204edb3ca6db None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.492930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5d21-18ca-4f12-8e2f-204edb3ca6db None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.493207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5d21-18ca-4f12-8e2f-204edb3ca6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.511516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6f0b5d21-18ca-4f12-8e2f-204edb3ca6db None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:42.512742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6f0b5d21-18ca-4f12-8e2f-204edb3ca6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:42.568699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4a76e-f455-4914-a841-67bf2ae009b9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.578210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 56/109] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:42.579016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5d21-18ca-4f12-8e2f-204edb3ca6db None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.596262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.605695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.606312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.606593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.606859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.677868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.679279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.692219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5d21-18ca-4f12-8e2f-204edb3ca6db None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.704485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.735600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.736214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.736533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.736805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.737464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:42.737775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.739136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.747831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 57/110] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:42.748468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.785486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 54/111] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:42.786321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.836521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.863118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc93e-e001-49ea-90b8-87820fb6d36b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.884499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0b53b-8e6f-4aef-a77b-772e6d0fe941 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:42.895571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.896196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.896509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.896778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.897877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.899302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:42.900907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 58/112] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:42.901661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:42.907830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:42.919139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:42.919746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:42.920025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:42.920329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:42.997203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:42.998561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.000312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8754-7e28-432a-a73e-9a90052a6b73 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.033584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.034207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.034487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.034759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.035869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.037354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.043971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 59/113] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:43.043971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.051382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.097911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 55/114] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:43.098743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.121980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.143469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac72aaa-b7af-40f6-8ef0-46b9912cf40a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.176605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.177233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.177511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.177984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] PATH_INFO: `/v3/users/b3d5364c4bfc467ba54c1022c639c48c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.179115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d5364c4bfc467ba54c1022c639c48c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.180596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.182672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 60/115] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/users/b3d5364c4bfc467ba54c1022c639c48c => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:43.183431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.200026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f97e6-e3e6-4ee2-b73d-be5519c3675f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.231900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.232552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.232831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.233098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.234210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.235629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.237325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 56/116] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:43.237998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.260547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.282851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471189e3-c5f0-466b-b6ea-643a2beb0a64 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.315640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.316290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.316570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.316839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.327238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:43.327554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:43.332757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:43.333650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:43.334395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 61/117] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:43.335022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.341302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f48ecd-3f88-4041-8077-05dc0ab29bb6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.374911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.375521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.375799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.376072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.376759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.378200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.406300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 57/118] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:10:43 2022] POST /identity/v3/projects => generated 302 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:43.407109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.435321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7f55-c04a-43e2-a69f-dd83aa88f530 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.471245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.471874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.472155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.472457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.473192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=dcc09680cf024f28a65aaca40b00c00f, user.id=299712f6a5bb49b986da988b00e6dbe7) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:43.473509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.474975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.482208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 62/119] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/role_assignments?user.id=299712f6a5bb49b986da988b00e6dbe7&scope.project.id=dcc09680cf024f28a65aaca40b00c00f&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:43.482887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.495835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.553692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788dd03-f3c1-4046-9d61-1ac174216dbd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.593983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.594650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.594931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.595205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] PATH_INFO: `/v3/projects/922bd7e9333c4789bb68ed75fa5cef9e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.596627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] RBAC: Authorizing `identity:get_project(project_id=922bd7e9333c4789bb68ed75fa5cef9e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.598123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.606462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 58/120] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/projects/922bd7e9333c4789bb68ed75fa5cef9e => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:43.606462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.629305 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.660339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f13d8-8ce1-4668-bbcc-7e3974ddc63d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.707632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.708269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.708557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.708829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.738283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606209f5-a023-4e37-a81b-3fb8cc4f57ee None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.781175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.781793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.782074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.782351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.783077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] RBAC: Adding query filter params (role.id=fc711bb446524efbbb3ee062f6ff2702, scope.project.id=dcc09680cf024f28a65aaca40b00c00f, user.id=b3d5364c4bfc467ba54c1022c639c48c) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:43.783417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.784883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.791575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.792694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 59/121] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/role_assignments?user.id=b3d5364c4bfc467ba54c1022c639c48c&scope.project.id=dcc09680cf024f28a65aaca40b00c00f&role.id=fc711bb446524efbbb3ee062f6ff2702 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:43.793495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.793879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.801173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.889615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:43.920208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76773b-397f-4467-b262-cbde7a15445a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:43.941832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 63/122] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:43.943496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:43.958538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:43.959526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:43.959977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:43.960426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:43.961296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:43.961779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:43.964116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:43.974066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 60/123] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:43.975260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:44.074001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a64707-3f83-41fc-997e-1f43ad476692 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:44.075563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:44.095137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645efc37-4ab2-4c8c-93ce-0c4cf7f2e6c8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:44.114603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:44.115210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:44.115487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:44.115758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:44.116493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] RBAC: Adding query filter params (role.id=111b31616dc54e43a9000ee72a3ac5b8, scope.project.id=922bd7e9333c4789bb68ed75fa5cef9e, user.id=b3d5364c4bfc467ba54c1022c639c48c) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:44.117220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:44.118660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:44.124735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 64/124] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/role_assignments?user.id=b3d5364c4bfc467ba54c1022c639c48c&scope.project.id=922bd7e9333c4789bb68ed75fa5cef9e&role.id=111b31616dc54e43a9000ee72a3ac5b8 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:44.126208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:44.130798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:44.131455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:44.131732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:44.131999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:44.133314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:44.134838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:44.140767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 61/125] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:43 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:44.144441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:44.155903 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:44.256755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca98e1-4d7c-4d3a-9c1c-76f2c62878ae None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:44.261584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d14dd4-269d-4df5-a0f4-f9f034b7d8bf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:44.301820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:44.302052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:44.302623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:44.302903 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:44.303181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] PATH_INFO: `/v3/users/dad6be2e509f44b3804aa47545306a85` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:44.303578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:44.304191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:44.304655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:44.305702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] RBAC: Adding query filter params (role.id=111b31616dc54e43a9000ee72a3ac5b8, scope.project.id=dcc09680cf024f28a65aaca40b00c00f, user.id=b3d5364c4bfc467ba54c1022c639c48c) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:44.306234 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:44.307987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:44.315139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 62/126] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:44 2022] GET /identity/v3/role_assignments?user.id=b3d5364c4bfc467ba54c1022c639c48c&scope.project.id=dcc09680cf024f28a65aaca40b00c00f&role.id=111b31616dc54e43a9000ee72a3ac5b8 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:44.331670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] RBAC: Authorizing `identity:get_user(user_id=dad6be2e509f44b3804aa47545306a85)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:44.333561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:44.335522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 65/127] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:44 2022] GET /identity/v3/users/dad6be2e509f44b3804aa47545306a85 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:44.341780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:44.428614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:44.466285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa8aea-1934-4ac7-9eb4-de5234031c43 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:44.503169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24852f6-59ad-41a4-aadf-adca53aa5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:44.503358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24852f6-59ad-41a4-aadf-adca53aa5e71 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:44.503557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24852f6-59ad-41a4-aadf-adca53aa5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:44.504268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24852f6-59ad-41a4-aadf-adca53aa5e71 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:44.505109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24852f6-59ad-41a4-aadf-adca53aa5e71 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:44.507186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24852f6-59ad-41a4-aadf-adca53aa5e71 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:44.508444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 63/128] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:44 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:44.514473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:44.541805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:44.621257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf8367-fea9-4f14-9978-29e900619f61 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:44.653937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a2c81-d4e4-4e37-95de-ce4368972878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:44.654589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a2c81-d4e4-4e37-95de-ce4368972878 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:44.654864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a2c81-d4e4-4e37-95de-ce4368972878 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:44.655134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a2c81-d4e4-4e37-95de-ce4368972878 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/dad6be2e509f44b3804aa47545306a85/roles/184d8fb55ab246ac8a358668dd43bb35` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:44.657021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a2c81-d4e4-4e37-95de-ce4368972878 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcc09680cf024f28a65aaca40b00c00f, user_id=dad6be2e509f44b3804aa47545306a85, role_id=184d8fb55ab246ac8a358668dd43bb35)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:44.658456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a2c81-d4e4-4e37-95de-ce4368972878 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:44.670294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 66/129] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:44 2022] PUT /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f/users/dad6be2e509f44b3804aa47545306a85/roles/184d8fb55ab246ac8a358668dd43bb35 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:10:45.671399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421151f-e5b7-47ac-b379-23a2af031376 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:45.671846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421151f-e5b7-47ac-b379-23a2af031376 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:45.671925 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421151f-e5b7-47ac-b379-23a2af031376 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:45.673461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 64/130] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:45.678361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b7fbb-7fcf-4c4c-a256-91cb68e1b437 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:45.678730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b7fbb-7fcf-4c4c-a256-91cb68e1b437 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:45.679009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b7fbb-7fcf-4c4c-a256-91cb68e1b437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:45.697152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-044b7fbb-7fcf-4c4c-a256-91cb68e1b437 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:45.698334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-044b7fbb-7fcf-4c4c-a256-91cb68e1b437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:45.762102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 67/131] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:46.037648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421151f-e5b7-47ac-b379-23a2af031376 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:46.134957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421151f-e5b7-47ac-b379-23a2af031376 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:46.167276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed05c15-454b-408f-a7d3-e45a0329c303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:46.167891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed05c15-454b-408f-a7d3-e45a0329c303 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:46.168171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed05c15-454b-408f-a7d3-e45a0329c303 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:46.168461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed05c15-454b-408f-a7d3-e45a0329c303 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:46.169545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed05c15-454b-408f-a7d3-e45a0329c303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:46.170980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed05c15-454b-408f-a7d3-e45a0329c303 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:46.172711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 65/132] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:46.177990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b7fbb-7fcf-4c4c-a256-91cb68e1b437 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:46.192475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:46.275072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b7fbb-7fcf-4c4c-a256-91cb68e1b437 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:46.307681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c4687-e88b-49ec-b8a1-028f56f2ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:46.308219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4687-e88b-49ec-b8a1-028f56f2ef62 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:46.308519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4687-e88b-49ec-b8a1-028f56f2ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:46.308789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4687-e88b-49ec-b8a1-028f56f2ef62 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:46.309427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c4687-e88b-49ec-b8a1-028f56f2ef62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:46.310854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c4687-e88b-49ec-b8a1-028f56f2ef62 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:46.332047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 68/133] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:10:46 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:46.799797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc593db-fe52-4e96-8d9d-523339acc319 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:46.800304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc593db-fe52-4e96-8d9d-523339acc319 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:46.800585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc593db-fe52-4e96-8d9d-523339acc319 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:46.801960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 66/134] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:46.807954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b8f0-d256-4f30-b782-bf59575099f0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:46.808824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b8f0-d256-4f30-b782-bf59575099f0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:46.809284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b8f0-d256-4f30-b782-bf59575099f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:46.827801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2433b8f0-d256-4f30-b782-bf59575099f0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:46.829335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2433b8f0-d256-4f30-b782-bf59575099f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:46.897408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:46.902674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 69/135] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:47.174252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc593db-fe52-4e96-8d9d-523339acc319 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:47.183766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:47.280297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc593db-fe52-4e96-8d9d-523339acc319 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:47.316814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:47.317838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:47.318113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:47.318380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:47.329640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:47.334973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:47.335803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:47.336975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 67/136] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:47.342551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433b8f0-d256-4f30-b782-bf59575099f0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:47.445687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433b8f0-d256-4f30-b782-bf59575099f0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:47.483495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:47.484109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:47.484416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:47.484687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:47.485340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:47.485637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:47.487011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:47.495383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 70/137] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:47.501107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:47.568935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:47.604817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060a373-64ea-4512-b342-541a50387c8b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:47.640101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:47.641019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:47.641294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:47.641560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] PATH_INFO: `/v3/users/dad6be2e509f44b3804aa47545306a85` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:47.642683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] RBAC: Authorizing `identity:get_user(user_id=dad6be2e509f44b3804aa47545306a85)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:47.644109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:47.645862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 68/138] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:47 2022] GET /identity/v3/users/dad6be2e509f44b3804aa47545306a85 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:47.651267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:47.715587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:47.754569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d62cf1-e408-4a09-a89c-07a51598701e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:47.788667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:47.789276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:47.789552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:47.789823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:47.857299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:47.858757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:47.918759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:47.959182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 71/139] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:47.965840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:48.066426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a22fe-63d6-4a9c-b17b-11cb8ec73903 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:48.099392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c769b1-15a1-48a6-9d89-3aee67d17e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:48.100019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c769b1-15a1-48a6-9d89-3aee67d17e86 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.100322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c769b1-15a1-48a6-9d89-3aee67d17e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.100596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c769b1-15a1-48a6-9d89-3aee67d17e86 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.101713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c769b1-15a1-48a6-9d89-3aee67d17e86 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:48.103782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c769b1-15a1-48a6-9d89-3aee67d17e86 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:48.105423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 69/140] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:47 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:48.111234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:48.210691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3611eb-7c0f-46a5-8b19-ab89fc8f488b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:48.246475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfe206-522f-4a78-afdd-b1e3a54e16a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:48.247016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfe206-522f-4a78-afdd-b1e3a54e16a8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.247298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfe206-522f-4a78-afdd-b1e3a54e16a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.247571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfe206-522f-4a78-afdd-b1e3a54e16a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.248322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfe206-522f-4a78-afdd-b1e3a54e16a8 None admin] RBAC: Adding query filter params (role.id=184d8fb55ab246ac8a358668dd43bb35, scope.project.id=dcc09680cf024f28a65aaca40b00c00f, user.id=dad6be2e509f44b3804aa47545306a85) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:48.248629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfe206-522f-4a78-afdd-b1e3a54e16a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:48.250053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfe206-522f-4a78-afdd-b1e3a54e16a8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:48.256294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 72/141] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:48 2022] GET /identity/v3/role_assignments?user.id=dad6be2e509f44b3804aa47545306a85&scope.project.id=dcc09680cf024f28a65aaca40b00c00f&role.id=184d8fb55ab246ac8a358668dd43bb35 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:48.606464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240dc8d-f2d0-43b9-a799-eb57c3691aeb None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.606675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240dc8d-f2d0-43b9-a799-eb57c3691aeb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.607130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240dc8d-f2d0-43b9-a799-eb57c3691aeb None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.608529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 70/142] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:48.616344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956d249-ceb1-41c1-be74-148ae0b2fd76 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.616917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956d249-ceb1-41c1-be74-148ae0b2fd76 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.617201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956d249-ceb1-41c1-be74-148ae0b2fd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.635439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9956d249-ceb1-41c1-be74-148ae0b2fd76 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:48.636720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9956d249-ceb1-41c1-be74-148ae0b2fd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:48.661459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:48.711493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 73/143] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:48.763626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812560-57bd-4054-812c-9e2c4a94a25a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.763910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812560-57bd-4054-812c-9e2c4a94a25a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.764402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812560-57bd-4054-812c-9e2c4a94a25a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.765991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 71/144] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:48.771238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b3ab1-9c05-4405-98c7-bca5f0eadd40 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.771516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b3ab1-9c05-4405-98c7-bca5f0eadd40 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.771791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b3ab1-9c05-4405-98c7-bca5f0eadd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.789481 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-638b3ab1-9c05-4405-98c7-bca5f0eadd40 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:48.791179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-638b3ab1-9c05-4405-98c7-bca5f0eadd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:48.824991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62e796-acad-4a26-b76f-488f8e09fb3f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.825278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62e796-acad-4a26-b76f-488f8e09fb3f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.825721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62e796-acad-4a26-b76f-488f8e09fb3f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.827044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 72/145] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:48.832038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf42ab-a13b-426e-8bc9-27aa8baaeb7d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:48.832528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf42ab-a13b-426e-8bc9-27aa8baaeb7d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:48.833004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf42ab-a13b-426e-8bc9-27aa8baaeb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:48.839381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:48.852088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4bbf42ab-a13b-426e-8bc9-27aa8baaeb7d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:48.854043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4bbf42ab-a13b-426e-8bc9-27aa8baaeb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:48.863125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 74/146] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:48.920270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 73/147] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:48.997505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b3ab1-9c05-4405-98c7-bca5f0eadd40 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.079764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.095578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b3ab1-9c05-4405-98c7-bca5f0eadd40 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.120683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf42ab-a13b-426e-8bc9-27aa8baaeb7d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.130167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.130801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.131074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.131344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.142214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:49.146994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:49.147856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:49.149099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 75/148] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:10:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:49.154280 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.223148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf42ab-a13b-426e-8bc9-27aa8baaeb7d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.252601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.254837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f276e-fccb-4282-8ee0-0b22d5feca1e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.257952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.258573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.258848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.259193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.269230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:49.274005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:49.275540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 74/149] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:10:49.276365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.292557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.293166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.293438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.293706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.294366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:49.294669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.296026 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.304021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 76/150] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:49.305741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.316139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.376270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0bca-a4de-48cd-baf5-1ab7e037ac23 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.401963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef4695-817e-40bb-956a-85eeb2abf32a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.409295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.409912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.410179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.410449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.411536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.413000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.414546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 75/151] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:49.415340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.434999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.435597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.435871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.436136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.436818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:49.437113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.438454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.446279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 77/152] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:49.448047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.475203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.515404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad92a5a-b903-4e65-a721-0682335cfd95 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.516839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.547637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd259f7d-52eb-430d-951a-7110dcc1b1de None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.549083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.549703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.549981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.550247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] PATH_INFO: `/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.563034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] RBAC: Authorizing `identity:get_user(user_id=934b52f25d3d4cb4b2218b1d8f2ad845)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.564498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.566075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 76/153] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:49.566823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.585059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.585643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.585918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.586185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.586815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.588845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.610016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 78/154] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:10:49 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:49.610016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.677493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7f637-90d6-4c16-9b9d-70d8fffab830 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.695955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.715188 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.715836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.716115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.716424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] PATH_INFO: `/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.717584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] RBAC: Authorizing `identity:get_user(user_id=934b52f25d3d4cb4b2218b1d8f2ad845)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.719051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.725266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 77/155] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:49.726401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.734597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4072198-96b5-42d7-8753-cb94c5ccaca9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.762615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.775680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.776343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.776618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.776888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.778005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.779459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.781434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 79/156] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:49.786951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.839692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8db9c-1f7b-48f5-955d-52503bca357f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.880770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.881338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.881611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.881879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.884024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.888043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.889086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 78/157] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:49 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:49.897099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:49.915902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67a549-0a92-400f-889c-7a06c739ae46 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:49.965945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bcf7-9aab-44b6-ab03-f163db7c3015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:49.966617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bcf7-9aab-44b6-ab03-f163db7c3015 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:49.966897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bcf7-9aab-44b6-ab03-f163db7c3015 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:49.967165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bcf7-9aab-44b6-ab03-f163db7c3015 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70/users/934b52f25d3d4cb4b2218b1d8f2ad845/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:49.972005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:49.972162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593bcf7-9aab-44b6-ab03-f163db7c3015 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a40440cb403480ca7194909d0e78c70, user_id=934b52f25d3d4cb4b2218b1d8f2ad845, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:49.973871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593bcf7-9aab-44b6-ab03-f163db7c3015 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:49.991688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 80/158] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:49 2022] PUT /identity/v3/projects/3a40440cb403480ca7194909d0e78c70/users/934b52f25d3d4cb4b2218b1d8f2ad845/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:10:50.032690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b780ff4-6ed2-4196-a488-14d92bda10e1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:50.081615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5115f-c050-4afd-b43d-c656037c0d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:50.082680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5115f-c050-4afd-b43d-c656037c0d73 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:50.083160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5115f-c050-4afd-b43d-c656037c0d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:50.083629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5115f-c050-4afd-b43d-c656037c0d73 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70/users/934b52f25d3d4cb4b2218b1d8f2ad845/roles/fc711bb446524efbbb3ee062f6ff2702` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:50.086284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5115f-c050-4afd-b43d-c656037c0d73 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a40440cb403480ca7194909d0e78c70, user_id=934b52f25d3d4cb4b2218b1d8f2ad845, role_id=fc711bb446524efbbb3ee062f6ff2702)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:50.088116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5115f-c050-4afd-b43d-c656037c0d73 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:50.103855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 79/159] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:49 2022] PUT /identity/v3/projects/3a40440cb403480ca7194909d0e78c70/users/934b52f25d3d4cb4b2218b1d8f2ad845/roles/fc711bb446524efbbb3ee062f6ff2702 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:10:51.893683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26019a7f-8499-49d5-8cd4-c91071b9749b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:51.894662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26019a7f-8499-49d5-8cd4-c91071b9749b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:51.895159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26019a7f-8499-49d5-8cd4-c91071b9749b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:51.897143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 81/160] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:51.903128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2f2f-b529-48e0-adc9-646a7c13843a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:51.903929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2f2f-b529-48e0-adc9-646a7c13843a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:51.904445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2f2f-b529-48e0-adc9-646a7c13843a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:51.923710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-87ca2f2f-b529-48e0-adc9-646a7c13843a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:51.924922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87ca2f2f-b529-48e0-adc9-646a7c13843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:51.965163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdae401-adce-4488-9a8a-ed4e8f983cac None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:51.965765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdae401-adce-4488-9a8a-ed4e8f983cac None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:51.966250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdae401-adce-4488-9a8a-ed4e8f983cac None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:51.967797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 82/161] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:10:51.973093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ad6a-96c6-4678-84fc-3128005d166c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:51.973610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ad6a-96c6-4678-84fc-3128005d166c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:51.974074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ad6a-96c6-4678-84fc-3128005d166c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:51.992484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0555ad6a-96c6-4678-84fc-3128005d166c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:51.993989 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0555ad6a-96c6-4678-84fc-3128005d166c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:51.994779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 80/162] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:52.049927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:52.070372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 83/163] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:52.236070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bac29-9eb5-42c7-95e3-14f5567997b3 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.236879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bac29-9eb5-42c7-95e3-14f5567997b3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.237446 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bac29-9eb5-42c7-95e3-14f5567997b3 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.239001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 81/164] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:52.245272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550898d0-4394-4718-844e-c2f377c24425 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.245692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550898d0-4394-4718-844e-c2f377c24425 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.245870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550898d0-4394-4718-844e-c2f377c24425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.253719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736477-2bbd-47b1-9d67-00ac18abe9ab None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.254951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736477-2bbd-47b1-9d67-00ac18abe9ab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.254951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736477-2bbd-47b1-9d67-00ac18abe9ab None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.255847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 82/165] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:52.262109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b31a9-0a74-458a-9bf1-56d15d856c73 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.262109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b31a9-0a74-458a-9bf1-56d15d856c73 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.262333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b31a9-0a74-458a-9bf1-56d15d856c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.268561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-550898d0-4394-4718-844e-c2f377c24425 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:52.269851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-550898d0-4394-4718-844e-c2f377c24425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:52.282176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d30b31a9-0a74-458a-9bf1-56d15d856c73 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:52.283326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d30b31a9-0a74-458a-9bf1-56d15d856c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:52.327592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:52.342004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 84/166] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:52.342833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550898d0-4394-4718-844e-c2f377c24425 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.358329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 83/167] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:52.359372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b31a9-0a74-458a-9bf1-56d15d856c73 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.452578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550898d0-4394-4718-844e-c2f377c24425 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:52.467174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:52.478854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b31a9-0a74-458a-9bf1-56d15d856c73 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:52.489994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:52.490880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.491366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.491848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.503073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:52.508162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:52.509162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:52.509927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:52.512164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 85/168] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:52.517202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:52.517202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.517202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.517202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.517471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:52.518301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:52.519044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.520193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 84/169] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:52.525096 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.571685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:52.621584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919277f7-f99c-4bf2-a54b-ca86ce3323a6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:52.633004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca4d9a-4588-475f-9220-5f89100f1daa None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:52.656017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:52.656648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.656914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.657178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.658108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:52.658403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:52.659756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:52.667211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:52.667806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.668075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.668378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.668488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 86/170] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:52.669024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:52.670430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:52.673437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.696887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 85/171] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:10:52 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:10:52.696887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.710100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:52.782441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32136c-9826-4672-9e5d-a91eaa849e82 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:52.811583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:52.816730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf36b8-1e6c-41dc-9867-8d90e378dfda None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:52.823032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:52.823682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.823958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.824250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.836317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:52.841881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:52.842855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:52.843677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 87/172] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:10:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:52.844371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.859920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:52.860973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:52.861248 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:52.861512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:52.873179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:52.877706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:52.878563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 86/173] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:10:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:52.880701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:52.951317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:52.957935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae306e-856e-4373-a4b7-2ddfcd3bed83 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.000683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c958ae-ab4f-441f-b8b9-8ba6440b0b2e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.008054 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.008721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.009026 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.009294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] PATH_INFO: `/v3/users/483e5c2e8ac4483dba3a8eeab570a300` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.027709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] RBAC: Authorizing `identity:get_user(user_id=483e5c2e8ac4483dba3a8eeab570a300)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.029296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.030876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 88/174] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:52 2022] GET /identity/v3/users/483e5c2e8ac4483dba3a8eeab570a300 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:53.036099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.049809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.050189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.050509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.050775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.051489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:53.051782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.053233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.063540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 87/175] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:53.069129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.097857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:53.156707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0b334-75cd-4376-8183-7df223aae166 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.199878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044e321-9a60-4363-b398-bf2c09d6605f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.202405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.203018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.204083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.204408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.205065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:53.205365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.206736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.217402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 89/176] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:10:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:53.220441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.247603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:53.255879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.256802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.257082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.257351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.259574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.261401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.268826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 88/177] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:53 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:53.268826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.362780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:53.369304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437fc6a-2f6e-4403-8a18-e21186f4b384 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.377204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37452f84-d1ef-47f5-91a6-28979dbd7593 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.416467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.417149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.417434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.417716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] PATH_INFO: `/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.418617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.418819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] RBAC: Authorizing `identity:get_user(user_id=934b52f25d3d4cb4b2218b1d8f2ad845)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.419651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.420127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.420248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.420770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] PATH_INFO: `/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.421843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 90/178] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:53 2022] GET /identity/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:53.426500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.426590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] RBAC: Authorizing `identity:get_user(user_id=934b52f25d3d4cb4b2218b1d8f2ad845)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.426590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.428157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 89/179] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:53 2022] GET /identity/v3/users/934b52f25d3d4cb4b2218b1d8f2ad845 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:53.433331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.529654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:53.542851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a64ba8-ed44-44a1-9265-a32a5062dae2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.549849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45632-6a6f-40f9-addd-b1a4a9e94aee None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.578444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.579103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.579383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.579659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.590484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.591450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.591926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.592473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70/users/483e5c2e8ac4483dba3a8eeab570a300/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.594637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a40440cb403480ca7194909d0e78c70, user_id=483e5c2e8ac4483dba3a8eeab570a300, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.597054 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.606910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:53.612395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 91/180] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:53 2022] PUT /identity/v3/projects/3a40440cb403480ca7194909d0e78c70/users/483e5c2e8ac4483dba3a8eeab570a300/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:10:53.622356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.708896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:53.712628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:53.842572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac5a38-2745-427c-8542-0c4e60ada2f0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:53.889669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:53.890466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:53.890748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:53.891271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:53.903127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 90/181] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:53.905405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:53.910680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:54.011854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:54.019581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:54.021121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:54.102127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7ccb-b3f8-442a-b7c7-4cbd698e5914 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:54.161030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:54.161586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:54.161865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:54.162136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:54.163282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:54.164779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:54.167785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 91/182] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:53 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:54.190037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:54.240986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 92/183] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:54.256851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:54.309739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:54.363284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ccac0-7609-4873-b1bb-3e25e94ea214 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:54.400496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:54.401131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:54.401589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:54.401867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:54.403317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:54.404850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:54.406659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 93/184] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:54 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:54.413326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de7756-efeb-411e-9fb5-877dc9cc47e5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:54.415145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:54.465888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:54.492834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef5d58-f76c-4d81-832e-45183c359cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:54.493500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5d58-f76c-4d81-832e-45183c359cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:54.493780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5d58-f76c-4d81-832e-45183c359cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:54.494045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5d58-f76c-4d81-832e-45183c359cc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:54.494777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef5d58-f76c-4d81-832e-45183c359cc7 None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=3a40440cb403480ca7194909d0e78c70, user.id=934b52f25d3d4cb4b2218b1d8f2ad845) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:54.495084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef5d58-f76c-4d81-832e-45183c359cc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:54.497680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef5d58-f76c-4d81-832e-45183c359cc7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:54.514170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 92/185] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:54 2022] GET /identity/v3/role_assignments?user.id=934b52f25d3d4cb4b2218b1d8f2ad845&scope.project.id=3a40440cb403480ca7194909d0e78c70&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:54.598860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0a39-9093-4f8c-8971-60a7589a1073 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:54.628601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:54.678197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba6505-2928-4a17-8d31-875123407088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:54.678744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba6505-2928-4a17-8d31-875123407088 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:54.679017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba6505-2928-4a17-8d31-875123407088 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:54.679282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba6505-2928-4a17-8d31-875123407088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:54.680009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba6505-2928-4a17-8d31-875123407088 None admin] RBAC: Adding query filter params (role.id=fc711bb446524efbbb3ee062f6ff2702, scope.project.id=3a40440cb403480ca7194909d0e78c70, user.id=934b52f25d3d4cb4b2218b1d8f2ad845) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:54.680342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba6505-2928-4a17-8d31-875123407088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:54.682343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba6505-2928-4a17-8d31-875123407088 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:54.693914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 94/186] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:54 2022] GET /identity/v3/role_assignments?user.id=934b52f25d3d4cb4b2218b1d8f2ad845&scope.project.id=3a40440cb403480ca7194909d0e78c70&role.id=fc711bb446524efbbb3ee062f6ff2702 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:55.126002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83532-5b76-4038-9ae0-a0fd75a8cf3a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.126496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83532-5b76-4038-9ae0-a0fd75a8cf3a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.126569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83532-5b76-4038-9ae0-a0fd75a8cf3a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.128018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 93/187] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:55.133047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ab53a-657f-4b51-87b4-274dda607d5e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.133452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ab53a-657f-4b51-87b4-274dda607d5e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.133606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ab53a-657f-4b51-87b4-274dda607d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.137417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0507b2-129b-4778-b56c-857dab1c8a27 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.137697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0507b2-129b-4778-b56c-857dab1c8a27 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.137963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0507b2-129b-4778-b56c-857dab1c8a27 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.139328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 94/188] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:10:55.144298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cb6f4-66a8-4699-8d0a-f92fc0f4fb08 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.144578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cb6f4-66a8-4699-8d0a-f92fc0f4fb08 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.144847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cb6f4-66a8-4699-8d0a-f92fc0f4fb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.152526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e65ab53a-657f-4b51-87b4-274dda607d5e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:55.153710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e65ab53a-657f-4b51-87b4-274dda607d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:55.162644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e43cb6f4-66a8-4699-8d0a-f92fc0f4fb08 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:55.163794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e43cb6f4-66a8-4699-8d0a-f92fc0f4fb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:55.223888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 95/189] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:55.241592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 95/190] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:55.497309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ab53a-657f-4b51-87b4-274dda607d5e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:55.508085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cb6f4-66a8-4699-8d0a-f92fc0f4fb08 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:55.598395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ab53a-657f-4b51-87b4-274dda607d5e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:55.606006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:55.622847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cb6f4-66a8-4699-8d0a-f92fc0f4fb08 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:55.634920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:55.635746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.636023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.636322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.641698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:55.647787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:55.653134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:55.654006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:55.654810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 96/191] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:55.659852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:55.661912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:55.662549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.662936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.663104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.664859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:55.665660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:55.667362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 96/192] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:55.672437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:55.765091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff592-dfd8-4309-bfc7-862fcfbfa764 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:55.781771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024b59-b1f7-4842-8f4c-42ee37927dc3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:55.799246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:55.799846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.800115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.800400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.801063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:55.801357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:55.802707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:55.811210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 97/193] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:55.816714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:55.821032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:55.822169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.822445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.822715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.823370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:55.824824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:55.847073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 97/194] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:10:55 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:10:55.847431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:55.931987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e9b4c-94ac-404f-8af4-3a2c3808a253 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:55.971817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659f18b-a0b8-4c7e-ae35-d44110331900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:55.972515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659f18b-a0b8-4c7e-ae35-d44110331900 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:55.972858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659f18b-a0b8-4c7e-ae35-d44110331900 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:55.973130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659f18b-a0b8-4c7e-ae35-d44110331900 None admin] PATH_INFO: `/v3/users/0a8193b6e73948159c05ea989fb472fc` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:55.988456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659f18b-a0b8-4c7e-ae35-d44110331900 None admin] RBAC: Authorizing `identity:get_user(user_id=0a8193b6e73948159c05ea989fb472fc)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:55.989956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659f18b-a0b8-4c7e-ae35-d44110331900 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:55.996523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 98/195] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:55 2022] GET /identity/v3/users/0a8193b6e73948159c05ea989fb472fc => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:55.998491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:56.045416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:56.064359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5dcc5-cfbc-41fb-bb08-29c8334d8631 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:56.064648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5dcc5-cfbc-41fb-bb08-29c8334d8631 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:56.064919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5dcc5-cfbc-41fb-bb08-29c8334d8631 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:56.066267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 99/196] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:10:56.071355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfffc9-21ea-4ea4-8356-e03da47e591d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:56.072612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfffc9-21ea-4ea4-8356-e03da47e591d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:56.072612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfffc9-21ea-4ea4-8356-e03da47e591d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:56.092112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-fadfffc9-21ea-4ea4-8356-e03da47e591d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:56.093540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fadfffc9-21ea-4ea4-8356-e03da47e591d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:56.112961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeb4f0-f89a-4de1-a18a-b7d8b7cf2bc5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:56.154706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:56.155425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:56.155627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:56.155899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:56.157151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:56.158664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:56.160702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 98/197] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:55 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:56.166292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:56.178677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 100/198] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:56.285178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0c828-94d6-4af1-971d-8546e8191c50 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:56.297076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:56.325363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:56.326030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:56.326306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:56.326577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70/users/0a8193b6e73948159c05ea989fb472fc/roles/184d8fb55ab246ac8a358668dd43bb35` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:56.328528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] RBAC: Authorizing `identity:create_grant(project_id=3a40440cb403480ca7194909d0e78c70, user_id=0a8193b6e73948159c05ea989fb472fc, role_id=184d8fb55ab246ac8a358668dd43bb35)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:56.329963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:56.344810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 99/199] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:10:56 2022] PUT /identity/v3/projects/3a40440cb403480ca7194909d0e78c70/users/0a8193b6e73948159c05ea989fb472fc/roles/184d8fb55ab246ac8a358668dd43bb35 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:10:56.458146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfffc9-21ea-4ea4-8356-e03da47e591d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:56.478758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:56.572165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfffc9-21ea-4ea4-8356-e03da47e591d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:56.610732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:56.611358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:56.611631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:56.611896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:56.625466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:56.630267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:56.631150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:56.632056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 101/200] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:56.637195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:56.741868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3cc91-55a6-4eaa-8b55-030270c8a61b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:56.777717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:56.778337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:56.778617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:56.778952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:56.779625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:56.779926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:56.781326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:56.789790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 100/201] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:56.796060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:56.882834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:56.912766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3124adc-67cf-4bbf-9502-b64beacdbc50 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:56.949982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:56.950610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:56.950890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:56.951164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] PATH_INFO: `/v3/users/483e5c2e8ac4483dba3a8eeab570a300` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:56.952329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] RBAC: Authorizing `identity:get_user(user_id=483e5c2e8ac4483dba3a8eeab570a300)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:56.953851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:56.955723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 102/202] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:56 2022] GET /identity/v3/users/483e5c2e8ac4483dba3a8eeab570a300 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:56.961302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:56.995802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:57.073202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76383c57-ac8b-4652-b3e9-f246bba0baac None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:57.108563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:57.109175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:57.109454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:57.109726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:57.181683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:57.183078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:57.214416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:57.291411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 101/203] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:57.297059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:57.403621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a043f1-658e-402c-b41a-7f4d84914afb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:57.437616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda58379-5dd6-4720-bffc-4045039ef5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:57.438175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58379-5dd6-4720-bffc-4045039ef5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:57.438453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58379-5dd6-4720-bffc-4045039ef5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:57.438723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58379-5dd6-4720-bffc-4045039ef5c9 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:57.439844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda58379-5dd6-4720-bffc-4045039ef5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:57.441327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda58379-5dd6-4720-bffc-4045039ef5c9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:57.443262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 103/204] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:57 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:57.448911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:57.552416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b336-0e1c-4c3c-901f-670a462fbc92 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:57.583009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:57.589645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a4de9-6c04-4983-b877-5f54bc0ae168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:57.590326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4de9-6c04-4983-b877-5f54bc0ae168 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:57.590607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4de9-6c04-4983-b877-5f54bc0ae168 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:57.590882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a4de9-6c04-4983-b877-5f54bc0ae168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:57.591616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4de9-6c04-4983-b877-5f54bc0ae168 None admin] RBAC: Adding query filter params (role.id=111b31616dc54e43a9000ee72a3ac5b8, scope.project.id=3a40440cb403480ca7194909d0e78c70, user.id=483e5c2e8ac4483dba3a8eeab570a300) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:57.591924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4de9-6c04-4983-b877-5f54bc0ae168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:57.593397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a4de9-6c04-4983-b877-5f54bc0ae168 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:57.599941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 102/205] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:10:57 2022] GET /identity/v3/role_assignments?user.id=483e5c2e8ac4483dba3a8eeab570a300&scope.project.id=3a40440cb403480ca7194909d0e78c70&role.id=111b31616dc54e43a9000ee72a3ac5b8 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:58.154452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75b10-5dc9-4eda-860b-e48c8758287e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.154822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75b10-5dc9-4eda-860b-e48c8758287e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.154997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75b10-5dc9-4eda-860b-e48c8758287e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.156384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 104/206] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:58.161409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9fc6-009d-44fd-9841-989286d9221b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.161691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9fc6-009d-44fd-9841-989286d9221b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.161956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9fc6-009d-44fd-9841-989286d9221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.179979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d4dd9fc6-009d-44fd-9841-989286d9221b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:58.181748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d4dd9fc6-009d-44fd-9841-989286d9221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:58.253626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 103/207] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:58.329716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0e14c-2eb4-4144-85c1-9a6665357a92 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.330215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0e14c-2eb4-4144-85c1-9a6665357a92 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.330311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0e14c-2eb4-4144-85c1-9a6665357a92 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.331595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 105/208] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:10:58.336854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c45ed6-60fc-4188-ad30-d21cc11dd9e1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.337173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c45ed6-60fc-4188-ad30-d21cc11dd9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.337451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c45ed6-60fc-4188-ad30-d21cc11dd9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.356414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-59c45ed6-60fc-4188-ad30-d21cc11dd9e1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:58.357584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-59c45ed6-60fc-4188-ad30-d21cc11dd9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:58.424000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 104/209] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:10:58.521720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0e14c-2eb4-4144-85c1-9a6665357a92 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:58.537140 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:58.593196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8dce6-18c8-4c0a-a65a-75fa103a216b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.593587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8dce6-18c8-4c0a-a65a-75fa103a216b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.594114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8dce6-18c8-4c0a-a65a-75fa103a216b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.595666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 105/210] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:10:58.600883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb6d57-f0a5-4bdc-97cf-50f3d19dc188 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.601165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb6d57-f0a5-4bdc-97cf-50f3d19dc188 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.601436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb6d57-f0a5-4bdc-97cf-50f3d19dc188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.622937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1ffb6d57-f0a5-4bdc-97cf-50f3d19dc188 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:10:58.624740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1ffb6d57-f0a5-4bdc-97cf-50f3d19dc188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:10:58.633657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0e14c-2eb4-4144-85c1-9a6665357a92 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:58.669035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:58.669662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.670152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.670361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.678028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:58.683211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:58.688139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:58.689142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:10:58.689924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 106/211] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:58.694854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:58.698500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 106/212] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:10:58.701183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb6d57-f0a5-4bdc-97cf-50f3d19dc188 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:58.771179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:58.802730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb6d57-f0a5-4bdc-97cf-50f3d19dc188 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:58.806927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ca147-4af3-4798-9dc6-ae6d85dff508 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:58.836798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:58.837959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.838238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.838509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.839606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:58.842518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:58.842643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 107/213] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:10:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:58.845343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:58.846129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.846550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.846963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.847756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:58.847911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:58.848373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:58.849862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:58.860128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 107/214] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:58.865610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:58.907241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:58.954166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd6910-9ef3-47fb-a9c3-10702eaf6552 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:58.972664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db859-c114-464c-85f5-691ae2c18d5e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:58.988002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:58.988641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:58.988917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:58.989191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:58.989829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:58.991249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:59.007791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:59.008516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:59.008793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:59.009135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] PATH_INFO: `/v3/users/bf683e0f580842218a109ade15ba1848` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:59.011014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:59.011973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 108/215] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:10:58 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:10:59.012960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:59.027012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=bf683e0f580842218a109ade15ba1848)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:59.029188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:59.036250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 108/216] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:58 2022] GET /identity/v3/users/bf683e0f580842218a109ade15ba1848 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:10:59.038312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:59.136642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2915652-4253-4974-9717-ee9cd937a8a3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:59.148253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:59.156810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c95654-aada-4127-928a-61f0f742e5c5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:59.174198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:59.174861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:59.175123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:59.175392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:59.186809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:10:59.192108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:10:59.193044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:10:59.194134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 109/217] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:10:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:10:59.196190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:59.197408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:59.198372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:59.198372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:59.198654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] RBAC: Adding query filter params (role.id=111b31616dc54e43a9000ee72a3ac5b8, user.id=bf683e0f580842218a109ade15ba1848, scope.system=all) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:59.198959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:59.200061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:59.200426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:59.205768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 109/218] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Oct 27 13:10:59 2022] GET /identity/v3/role_assignments?user.id=bf683e0f580842218a109ade15ba1848&scope.system=all&role.id=111b31616dc54e43a9000ee72a3ac5b8 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:59.323184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15621e0e-dfc5-44f1-97a5-d65980209697 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:59.363657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:59.364388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:59.364670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:59.364941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:59.365624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:10:59.365925 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:59.367338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:59.378538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 110/219] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:10:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:59.384863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:59.445775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:59.497249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff0e34-5f3c-484a-ae82-447a0b9f365e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:59.537588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:59.538220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:59.538497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:59.538764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] PATH_INFO: `/v3/users/0a8193b6e73948159c05ea989fb472fc` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:59.539892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] RBAC: Authorizing `identity:get_user(user_id=0a8193b6e73948159c05ea989fb472fc)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:59.541894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:59.543476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 110/220] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:10:59 2022] GET /identity/v3/users/0a8193b6e73948159c05ea989fb472fc => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:10:59.549065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:59.583438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:59.658274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207eca74-8a7c-4cc8-9fb7-7b5311703170 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:10:59.697535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:10:59.698154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:10:59.698481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:10:59.698757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:10:59.772294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:10:59.773687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:10:59.805102 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:10:59.876600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 111/221] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:10:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:10:59.885543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:10:59.992794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5adf78-029e-479d-9577-fee3d53e70e9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:00.017809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:00.029934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f6fce-edea-4135-8241-2a9d96e6b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:00.030495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f6fce-edea-4135-8241-2a9d96e6b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:00.030774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f6fce-edea-4135-8241-2a9d96e6b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:00.031041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f6fce-edea-4135-8241-2a9d96e6b8b8 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:00.032166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f6fce-edea-4135-8241-2a9d96e6b8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:00.033634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f6fce-edea-4135-8241-2a9d96e6b8b8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:00.035556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 111/222] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:10:59 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:00.041365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:00.152359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aeb0c-dd10-4e57-a330-d5db26d1da02 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:00.182965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:00.188633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920d28a-77d2-4f13-8b98-667f66cf3c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:00.189907 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920d28a-77d2-4f13-8b98-667f66cf3c05 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:00.190382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920d28a-77d2-4f13-8b98-667f66cf3c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:00.190849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920d28a-77d2-4f13-8b98-667f66cf3c05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:00.191942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920d28a-77d2-4f13-8b98-667f66cf3c05 None admin] RBAC: Adding query filter params (role.id=184d8fb55ab246ac8a358668dd43bb35, scope.project.id=3a40440cb403480ca7194909d0e78c70, user.id=0a8193b6e73948159c05ea989fb472fc) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:00.192475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920d28a-77d2-4f13-8b98-667f66cf3c05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:00.194113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920d28a-77d2-4f13-8b98-667f66cf3c05 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:00.201290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 112/223] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:00 2022] GET /identity/v3/role_assignments?user.id=0a8193b6e73948159c05ea989fb472fc&scope.project.id=3a40440cb403480ca7194909d0e78c70&role.id=184d8fb55ab246ac8a358668dd43bb35 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:01.376645 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee31e3e-c99f-479e-a035-4c4aa5fc78ce None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.377191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee31e3e-c99f-479e-a035-4c4aa5fc78ce None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.377191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee31e3e-c99f-479e-a035-4c4aa5fc78ce None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.378689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 112/224] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:01.383761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbb026-94aa-4823-a7f5-c7d77ec16ded None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.384206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbb026-94aa-4823-a7f5-c7d77ec16ded None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.384491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbb026-94aa-4823-a7f5-c7d77ec16ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.404755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a4dbb026-94aa-4823-a7f5-c7d77ec16ded None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:01.405980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a4dbb026-94aa-4823-a7f5-c7d77ec16ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:01.421181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53834d2d-33f0-4e60-86e1-68a8c0b10794 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.421497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53834d2d-33f0-4e60-86e1-68a8c0b10794 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.421705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53834d2d-33f0-4e60-86e1-68a8c0b10794 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.423264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 113/225] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:01.428917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0147a-cfd0-43e3-915c-b39dbadef14b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.429209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0147a-cfd0-43e3-915c-b39dbadef14b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.429460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0147a-cfd0-43e3-915c-b39dbadef14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.450235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cfc0147a-cfd0-43e3-915c-b39dbadef14b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:01.451416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cfc0147a-cfd0-43e3-915c-b39dbadef14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:01.481490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 113/226] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:01.525244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 114/227] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:01.530705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f716c-05ec-4d37-b691-4a2904e1117b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.530989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f716c-05ec-4d37-b691-4a2904e1117b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.531256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f716c-05ec-4d37-b691-4a2904e1117b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.532757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 114/228] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:01.537547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41395b8-28c2-47b8-a5c7-1b86ed06cd70 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.537892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41395b8-28c2-47b8-a5c7-1b86ed06cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.538171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41395b8-28c2-47b8-a5c7-1b86ed06cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.555688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f41395b8-28c2-47b8-a5c7-1b86ed06cd70 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:01.556953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f41395b8-28c2-47b8-a5c7-1b86ed06cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:01.628470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 115/229] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:01.746541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f716c-05ec-4d37-b691-4a2904e1117b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:01.787650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41395b8-28c2-47b8-a5c7-1b86ed06cd70 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:01.841686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:01.854516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f716c-05ec-4d37-b691-4a2904e1117b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:01.880712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:01.890155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:01.890785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.891318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.891582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.895590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41395b8-28c2-47b8-a5c7-1b86ed06cd70 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:01.902520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:01.907336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:01.908381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:01.909185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 115/230] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:01.909832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:01.930244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:01.931387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:01.931662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:01.931936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:01.933043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:01.934461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:01.935995 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 116/231] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:01.936770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:02.011144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ac422-9ec0-46d1-b381-8cebac497f59 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.038628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6f57-e8fa-4520-b18f-7d8f43a746a0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.043476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.044103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.044423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.044696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.054485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:02.058417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:02.060373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:02.061270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:02.062072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 116/232] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:02.062737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:02.077763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.078373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.078646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.078911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.079584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:02.079878 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:02.081278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:02.089183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 117/233] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:11:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:02.092781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:02.107935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:02.160058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72165aec-0cd0-462c-abe4-aaa18464ef70 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.190208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4906ca-e6a3-4f3b-81e3-69d276e8f386 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.193106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.193715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.193994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.194267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] PATH_INFO: `/v3/groups` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.194891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:02.196379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:02.211820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 117/234] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Oct 27 13:11:02 2022] POST /identity/v3/groups => generated 229 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:02.213017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:02.245688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.246545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.247058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.247320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.248011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:02.248337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:02.249791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:02.263606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 118/235] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:11:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:02.264886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:02.317978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:02.329449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb7e43-4f78-497c-98bc-82e4d906a1b5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.336990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:02.369137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.370323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.370606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.371005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] PATH_INFO: `/v3/users/7e62753175b84fb481a1beca19cab0aa` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.382204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40bedd-3a32-45d5-aeb7-1d2b4f24afe4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.386138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] RBAC: Authorizing `identity:get_user(user_id=7e62753175b84fb481a1beca19cab0aa)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:02.387609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:02.389320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 118/236] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:11:02 2022] GET /identity/v3/users/7e62753175b84fb481a1beca19cab0aa => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:02.395507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:02.421838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.422397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.423781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.423781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] PATH_INFO: `/v3/users/bf683e0f580842218a109ade15ba1848` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.425140 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] RBAC: Authorizing `identity:get_user(user_id=bf683e0f580842218a109ade15ba1848)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:02.426384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:02.428494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 119/237] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:11:02 2022] GET /identity/v3/users/bf683e0f580842218a109ade15ba1848 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:02.433850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:02.512205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ad0d-d863-40fe-89ad-252f365eb496 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.538573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4823b6-07b1-4496-9dd0-83135f87d19c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:02.547120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90306df-7f55-45dc-b694-a434c39632e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.547720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90306df-7f55-45dc-b694-a434c39632e3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.547990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90306df-7f55-45dc-b694-a434c39632e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.548505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90306df-7f55-45dc-b694-a434c39632e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.548997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90306df-7f55-45dc-b694-a434c39632e3 None admin] RBAC: Adding query filter params (role.id=184d8fb55ab246ac8a358668dd43bb35, user.id=7e62753175b84fb481a1beca19cab0aa, scope.system=all) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:02.549295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90306df-7f55-45dc-b694-a434c39632e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:02.550723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90306df-7f55-45dc-b694-a434c39632e3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:02.552705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:02.564264 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 119/238] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Oct 27 13:11:02 2022] GET /identity/v3/role_assignments?user.id=7e62753175b84fb481a1beca19cab0aa&scope.system=all&role.id=184d8fb55ab246ac8a358668dd43bb35 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:02.584093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa15568-2908-4c7d-94a9-13bd1c450a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:02.584793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa15568-2908-4c7d-94a9-13bd1c450a15 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:02.585069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa15568-2908-4c7d-94a9-13bd1c450a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:02.585340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa15568-2908-4c7d-94a9-13bd1c450a15 None admin] PATH_INFO: `/v3/system/users/bf683e0f580842218a109ade15ba1848/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:02.586809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa15568-2908-4c7d-94a9-13bd1c450a15 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bf683e0f580842218a109ade15ba1848, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:02.588862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa15568-2908-4c7d-94a9-13bd1c450a15 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:02.600862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 120/239] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Oct 27 13:11:02 2022] PUT /identity/v3/system/users/bf683e0f580842218a109ade15ba1848/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:11:04.396286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f554f9-4e5c-46b6-8bf2-b140cf338ae9 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:04.396816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f554f9-4e5c-46b6-8bf2-b140cf338ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:04.396816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f554f9-4e5c-46b6-8bf2-b140cf338ae9 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:04.398235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 120/240] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:04.403941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb617f-6782-4439-b45d-1719b8075a1d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:04.404365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb617f-6782-4439-b45d-1719b8075a1d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:04.404551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb617f-6782-4439-b45d-1719b8075a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:04.423517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2ddb617f-6782-4439-b45d-1719b8075a1d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:04.424780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2ddb617f-6782-4439-b45d-1719b8075a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:04.489858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 121/241] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:04.708156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba174b-1fd0-4e5a-b3bb-dd2e48967af8 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:04.708462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba174b-1fd0-4e5a-b3bb-dd2e48967af8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:04.708662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba174b-1fd0-4e5a-b3bb-dd2e48967af8 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:04.710135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 121/242] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:04.715574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e8cd-3f6a-4fe1-9f9f-4cda30a367eb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:04.715860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e8cd-3f6a-4fe1-9f9f-4cda30a367eb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:04.716132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e8cd-3f6a-4fe1-9f9f-4cda30a367eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:04.733939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d9c9e8cd-3f6a-4fe1-9f9f-4cda30a367eb None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:04.735104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d9c9e8cd-3f6a-4fe1-9f9f-4cda30a367eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:04.753910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba174b-1fd0-4e5a-b3bb-dd2e48967af8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:04.783533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:04.805049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 122/243] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:04.806416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193ac33-dddc-441e-bf4c-6c839b967de4 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:04.806699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193ac33-dddc-441e-bf4c-6c839b967de4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:04.806968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193ac33-dddc-441e-bf4c-6c839b967de4 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:04.808266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 123/244] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:04.813515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b6fb-c468-430f-8425-6879bbbb094a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:04.813769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b6fb-c468-430f-8425-6879bbbb094a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:04.814035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b6fb-c468-430f-8425-6879bbbb094a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:04.832427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-aca3b6fb-c468-430f-8425-6879bbbb094a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:04.833585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aca3b6fb-c468-430f-8425-6879bbbb094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:04.862853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:04.862998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba174b-1fd0-4e5a-b3bb-dd2e48967af8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:04.902831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:04.903451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:04.903725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:04.903989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:04.905104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:04.906539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:04.912691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 122/245] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:04.912801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 124/246] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:04.913664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.007006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:05.014046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186215a3-d9c3-46bd-8d77-ae41ba003152 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.047106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.047704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.047976 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.048257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] PATH_INFO: `/v3/groups` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.048957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:05.050389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:05.072253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 123/247] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Oct 27 13:11:04 2022] POST /identity/v3/groups => generated 226 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:05.082442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b6fb-c468-430f-8425-6879bbbb094a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.177604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.194579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b6fb-c468-430f-8425-6879bbbb094a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.234567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.235243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.235586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.235832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.249747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:05.254640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:05.255503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:05.256380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 125/248] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:05.262237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.291861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:05.301121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693828f-fdab-4b8b-93ac-a254dc623828 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.343047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.343693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.343969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.344255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.357523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:05.361831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:05.362698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:05.363928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 124/249] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:05.369557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.375599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ab440-f1d0-47fc-af93-e43d6933284b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.390356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:05.418866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.419571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.419851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.420140 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.420911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:05.421206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:05.422569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:05.432825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 126/250] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:11:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:05.439745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.491259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd242ed-034c-443e-b4e7-5db3e558bdf3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.536928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.537645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.537920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.538183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.538853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:05.539149 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:05.541663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:05.554183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 125/251] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:11:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:05.559859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:05.560106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.577451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5bb99-60fc-4fd6-92be-9be3e1034752 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.627637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.628447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.628819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.629204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] PATH_INFO: `/v3/users/7e62753175b84fb481a1beca19cab0aa` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.630429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=7e62753175b84fb481a1beca19cab0aa)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:05.633777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:05.635756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 127/252] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:11:05 2022] GET /identity/v3/users/7e62753175b84fb481a1beca19cab0aa => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:05.641444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.671672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:05.697202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057daa5a-c6d5-4ecb-b975-2a902f341dc9 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.737958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.738605 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.738955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.739330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] PATH_INFO: `/v3/users/bf683e0f580842218a109ade15ba1848` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.740726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] RBAC: Authorizing `identity:get_user(user_id=bf683e0f580842218a109ade15ba1848)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:05.742422 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:05.744496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 126/253] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:11:05 2022] GET /identity/v3/users/bf683e0f580842218a109ade15ba1848 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:05.749762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:05.796364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b296a38-81fc-405a-b516-9e847f5bd3b6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.846574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bf75e-8ffa-4110-b518-7cca732bfc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.848108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bf75e-8ffa-4110-b518-7cca732bfc22 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.848570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bf75e-8ffa-4110-b518-7cca732bfc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.848982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bf75e-8ffa-4110-b518-7cca732bfc22 None admin] PATH_INFO: `/v3/system/users/7e62753175b84fb481a1beca19cab0aa/roles/184d8fb55ab246ac8a358668dd43bb35` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.850538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bf75e-8ffa-4110-b518-7cca732bfc22 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e62753175b84fb481a1beca19cab0aa, role_id=184d8fb55ab246ac8a358668dd43bb35)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:05.853775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bf75e-8ffa-4110-b518-7cca732bfc22 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:05.863298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 128/254] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Oct 27 13:11:05 2022] PUT /identity/v3/system/users/7e62753175b84fb481a1beca19cab0aa/roles/184d8fb55ab246ac8a358668dd43bb35 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:11:05.900701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1e4a6-a3c4-4cf5-8cf1-404b27a7a08a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:05.951109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7151af-ab1f-4bdd-97cb-96f17adc07a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:05.951927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7151af-ab1f-4bdd-97cb-96f17adc07a7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:05.952335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7151af-ab1f-4bdd-97cb-96f17adc07a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:05.952707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7151af-ab1f-4bdd-97cb-96f17adc07a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:05.953556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7151af-ab1f-4bdd-97cb-96f17adc07a7 None admin] RBAC: Adding query filter params (role.id=111b31616dc54e43a9000ee72a3ac5b8, user.id=bf683e0f580842218a109ade15ba1848, scope.system=all) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:05.953965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7151af-ab1f-4bdd-97cb-96f17adc07a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:05.955615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7151af-ab1f-4bdd-97cb-96f17adc07a7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:05.972812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 127/255] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Oct 27 13:11:05 2022] GET /identity/v3/role_assignments?user.id=bf683e0f580842218a109ade15ba1848&scope.system=all&role.id=111b31616dc54e43a9000ee72a3ac5b8 => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:08.106806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6f9ae-211d-4c8a-a09c-1ee34b01b820 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.107294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6f9ae-211d-4c8a-a09c-1ee34b01b820 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.107294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6f9ae-211d-4c8a-a09c-1ee34b01b820 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.108993 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 129/256] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:08.126309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd6ba-f983-4859-a5f7-cad44ff7d4fa None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.126800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd6ba-f983-4859-a5f7-cad44ff7d4fa None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.126870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd6ba-f983-4859-a5f7-cad44ff7d4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.136516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:08.150884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-971dd6ba-f983-4859-a5f7-cad44ff7d4fa None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:08.152089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-971dd6ba-f983-4859-a5f7-cad44ff7d4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:08.233804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 128/257] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 13:11:08.240812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74268351-d580-4dc4-98cf-aba673cb8980 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.241087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74268351-d580-4dc4-98cf-aba673cb8980 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.241409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74268351-d580-4dc4-98cf-aba673cb8980 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.242868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 130/258] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:08.248410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e98ac7-7bde-4114-8f91-0216bda2f149 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.248691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e98ac7-7bde-4114-8f91-0216bda2f149 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.248955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e98ac7-7bde-4114-8f91-0216bda2f149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.273203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b6e98ac7-7bde-4114-8f91-0216bda2f149 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:08.274383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b6e98ac7-7bde-4114-8f91-0216bda2f149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:08.286622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 129/259] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 13:11:08.318946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2ae01-f5cc-4a63-a998-51fb189e25d8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.319262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2ae01-f5cc-4a63-a998-51fb189e25d8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.319491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2ae01-f5cc-4a63-a998-51fb189e25d8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.320867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 131/260] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:08.326300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab2a0-5ddf-4a66-98eb-e399a7e2b9f5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.326581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab2a0-5ddf-4a66-98eb-e399a7e2b9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.326852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab2a0-5ddf-4a66-98eb-e399a7e2b9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.346133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-11cab2a0-5ddf-4a66-98eb-e399a7e2b9f5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:08.347306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-11cab2a0-5ddf-4a66-98eb-e399a7e2b9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:08.355624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 130/261] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 27 13:11:08.664462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2ae01-f5cc-4a63-a998-51fb189e25d8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.681891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2ae01-f5cc-4a63-a998-51fb189e25d8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.686647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.687465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.687910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.688368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.704300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:08.709706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:08.710656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:08.711453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:08.718787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 132/262] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:08.720145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cab2a0-5ddf-4a66-98eb-e399a7e2b9f5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.737627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cab2a0-5ddf-4a66-98eb-e399a7e2b9f5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.743839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.744560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.744839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.745162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.745834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:08.746131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.747908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:11:08.747908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:11:08.747908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:11:08.752828 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.764684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.767845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 131/263] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:08.778938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4c99c-18c3-407d-99ee-b47fb34697fd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.782896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.787390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.788273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.788697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.789105 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.796005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3c28e-ba63-4fe0-9947-0f9a312039e2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.796590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:08.805596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:08.810063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.810712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:08.810890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:08.811736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:08.812571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:08.813187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.813187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.813306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 133/264] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:08.813370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.830797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f0a83-2c7c-45f9-8a52-e9296a746fe7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.833599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:08.836455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.837677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.837962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.838214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:08.839670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:08.840523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:08.841316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:08.845093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 132/265] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:08.845238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.845784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.846980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:11:08.846980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:11:08.846980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:11:08.848197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.850569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 134/266] 10.222.0.48 () {56 vars in 1201 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:08.851366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.867855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e146e39-c8cb-450e-92bd-d99e1c15d511 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.873010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.873609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.873885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.874050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bc559-e578-43d0-b66a-6d9f7b7a25a6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.874223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.874890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:08.875187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.876183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:11:08.876183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:11:08.876183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:11:08.877103 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.887510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 135/267] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:08.888461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.890069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.890701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.890977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.891503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.892247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:08.892497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.894086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.898615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a3786-8289-489e-b8c6-2b4772b74c0d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.908773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.909397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.909665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.911537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.913871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.915995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 133/268] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:08.921040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.923089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.925922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 136/269] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:08.931030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.942956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ba27e-a8fa-4650-ac26-96aaec8c0858 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.950045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8560e2-4277-4ca4-9c9d-142767629ac3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.954932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.956666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.957001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.957269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.959347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.960457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.961827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.963045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:11:08.963045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:11:08.963045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:11:08.964298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.964534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.964856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f/groups/8bf7569f0b354ff099196a705c13d8be/roles/fc711bb446524efbbb3ee062f6ff2702` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.966859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 134/270] 10.222.0.48 () {56 vars in 1201 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:08.968883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc09680cf024f28a65aaca40b00c00f, group_id=8bf7569f0b354ff099196a705c13d8be, role_id=fc711bb446524efbbb3ee062f6ff2702)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.969899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:11:08.969899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:11:08.969899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:11:08.970894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.976484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.986620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a6b1-8c30-42c4-b051-39c483e2acd7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:08.991981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be275b13-bc2c-4b01-baf8-85f947e5231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:08.992716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be275b13-bc2c-4b01-baf8-85f947e5231e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:08.992981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be275b13-bc2c-4b01-baf8-85f947e5231e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:08.993387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be275b13-bc2c-4b01-baf8-85f947e5231e admin admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:08.994608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be275b13-bc2c-4b01-baf8-85f947e5231e admin admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:08.995027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 137/271] 10.222.0.48 () {58 vars in 1463 bytes} [Thu Oct 27 13:11:08 2022] PUT /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f/groups/8bf7569f0b354ff099196a705c13d8be/roles/fc711bb446524efbbb3ee062f6ff2702 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:08.995990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:08.996764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be275b13-bc2c-4b01-baf8-85f947e5231e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:08.998485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 135/272] 10.222.0.48 () {56 vars in 1201 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:08.999762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b69c7-9f39-40d1-8e0d-aff962a2be1e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.000042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b69c7-9f39-40d1-8e0d-aff962a2be1e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.000344 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b69c7-9f39-40d1-8e0d-aff962a2be1e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.001940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 136/273] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:09.007645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b69c7-9f39-40d1-8e0d-aff962a2be1e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.091369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f75e-9dc4-4a9d-9519-47d010b89ba1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.097836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4073f2-276d-4e2c-9df3-bfaa76d915dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.098576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b69c7-9f39-40d1-8e0d-aff962a2be1e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.099057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4073f2-276d-4e2c-9df3-bfaa76d915dd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.099578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4073f2-276d-4e2c-9df3-bfaa76d915dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.100037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4073f2-276d-4e2c-9df3-bfaa76d915dd admin admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.101373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4073f2-276d-4e2c-9df3-bfaa76d915dd admin admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.103540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.104193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4073f2-276d-4e2c-9df3-bfaa76d915dd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.104473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.104850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.105218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.106502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.106871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 138/274] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Oct 27 13:11:08 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:09.108574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d38f10-8919-4390-8449-473e1b02ff8e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.109012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.109125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d38f10-8919-4390-8449-473e1b02ff8e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.109534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d38f10-8919-4390-8449-473e1b02ff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.111486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 137/275] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Oct 27 13:11:09 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:09.116340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.127415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c59a-7097-48da-a3e4-12a0861046b9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.133689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.135103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.135474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.135873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70/groups/8bf7569f0b354ff099196a705c13d8be/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.138080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a40440cb403480ca7194909d0e78c70, group_id=8bf7569f0b354ff099196a705c13d8be, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.138841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a8d38f10-8919-4390-8449-473e1b02ff8e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:09.139389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:11:09.139389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:11:09.139389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:11:09.140566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a8d38f10-8919-4390-8449-473e1b02ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:09.142204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.150974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 139/276] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 27 13:11:09.152100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d38f10-8919-4390-8449-473e1b02ff8e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.162099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 138/277] 10.222.0.48 () {58 vars in 1463 bytes} [Thu Oct 27 13:11:09 2022] PUT /identity/v3/projects/3a40440cb403480ca7194909d0e78c70/groups/8bf7569f0b354ff099196a705c13d8be/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:09.163493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d38f10-8919-4390-8449-473e1b02ff8e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.229477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.230544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.231016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.231528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f/groups/8bf7569f0b354ff099196a705c13d8be/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.237240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc09680cf024f28a65aaca40b00c00f, group_id=8bf7569f0b354ff099196a705c13d8be, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.242448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.259210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 140/278] 10.222.0.48 () {58 vars in 1463 bytes} [Thu Oct 27 13:11:09 2022] PUT /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f/groups/8bf7569f0b354ff099196a705c13d8be/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:09.469117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.480140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff409736-0664-4d48-8017-31adf63600af admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.523263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:09.552590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.553239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.553517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.553786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.566145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:09.570990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:09.571923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:09.572791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 139/279] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Oct 27 13:11:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 104 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:09.578370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.595814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e6182-71f0-4b32-8345-dc2343d71f6d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.600732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.601659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.602110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.602554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.603933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:09.604445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.606325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.620714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 141/280] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Oct 27 13:11:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:09.626586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.641724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7660f-d0af-4d14-bbb7-fa24a680ebd2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.646434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.647349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.647821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.648303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.650153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.652381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.658749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 140/281] 10.222.0.48 () {56 vars in 1201 bytes} [Thu Oct 27 13:11:09 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:09.659799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.671075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337c9c8-33d1-4c00-8e50-9a1b19c877aa admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.682512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02378e47-7626-4959-b5a1-a2eab701b24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.683147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02378e47-7626-4959-b5a1-a2eab701b24c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.683473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02378e47-7626-4959-b5a1-a2eab701b24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.683744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02378e47-7626-4959-b5a1-a2eab701b24c admin admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.685945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02378e47-7626-4959-b5a1-a2eab701b24c admin admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.688158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02378e47-7626-4959-b5a1-a2eab701b24c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.691923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 142/282] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Oct 27 13:11:09 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:09.698375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:09.708936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a7c03-f3e8-4603-8967-3cc469b40423 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.709104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231d500-4a39-45bf-8ddf-670e3e7951d8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:09.709341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a7c03-f3e8-4603-8967-3cc469b40423 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.709528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a7c03-f3e8-4603-8967-3cc469b40423 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.710849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 143/283] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:09.718810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:09.719846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.720350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.720834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70/groups/8bf7569f0b354ff099196a705c13d8be/roles/fc711bb446524efbbb3ee062f6ff2702` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.723222 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a40440cb403480ca7194909d0e78c70, group_id=8bf7569f0b354ff099196a705c13d8be, role_id=fc711bb446524efbbb3ee062f6ff2702)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:09.725754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467d387-dcc9-451e-a74d-90378f6061e3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:09.726031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467d387-dcc9-451e-a74d-90378f6061e3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:09.726300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467d387-dcc9-451e-a74d-90378f6061e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:09.727105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:09.753130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 141/284] 10.222.0.48 () {58 vars in 1463 bytes} [Thu Oct 27 13:11:09 2022] PUT /identity/v3/projects/3a40440cb403480ca7194909d0e78c70/groups/8bf7569f0b354ff099196a705c13d8be/roles/fc711bb446524efbbb3ee062f6ff2702 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:09.757505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a467d387-dcc9-451e-a74d-90378f6061e3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:09.758768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a467d387-dcc9-451e-a74d-90378f6061e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:09.803416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:09.858929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 144/285] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:10.411433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:10.537679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b434-a0c6-4c52-bbb0-14619e56d25f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:10.582103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3c1c6-54c7-4a3b-af35-060474c668e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:10.583163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3c1c6-54c7-4a3b-af35-060474c668e3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:10.583638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3c1c6-54c7-4a3b-af35-060474c668e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:10.584098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3c1c6-54c7-4a3b-af35-060474c668e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:10.600837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3c1c6-54c7-4a3b-af35-060474c668e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-10e3c1c6-54c7-4a3b-af35-060474c668e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:10.606143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:10.607001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 27 13:11:10.607794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:10.610453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 142/286] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:10.618953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467d387-dcc9-451e-a74d-90378f6061e3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:10.714171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbae31-9119-4690-bdba-5a288eae5fbd None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:10.714411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbae31-9119-4690-bdba-5a288eae5fbd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:10.714804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbae31-9119-4690-bdba-5a288eae5fbd None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:10.716367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 143/287] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:10.721945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174006a-e0e7-42c2-ac62-4d9b524a0276 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:10.722449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174006a-e0e7-42c2-ac62-4d9b524a0276 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:10.722908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174006a-e0e7-42c2-ac62-4d9b524a0276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:10.743656 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e174006a-e0e7-42c2-ac62-4d9b524a0276 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:10.745231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e174006a-e0e7-42c2-ac62-4d9b524a0276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:10.757319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467d387-dcc9-451e-a74d-90378f6061e3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:10.761560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:10.822495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:10.823447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:10.823904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:10.824373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:10.825208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:10.825691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:10.827765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:10.831134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 144/288] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 27 13:11:10.844774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 145/289] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:11:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:10.850822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174006a-e0e7-42c2-ac62-4d9b524a0276 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:10.977794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174006a-e0e7-42c2-ac62-4d9b524a0276 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:11.018303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:11.018985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:11.019258 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:11.019521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] PATH_INFO: `/v3/users/7e62753175b84fb481a1beca19cab0aa` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:11.020688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] RBAC: Authorizing `identity:get_user(user_id=7e62753175b84fb481a1beca19cab0aa)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:11.022177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:11.023950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 145/290] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:11:10 2022] GET /identity/v3/users/7e62753175b84fb481a1beca19cab0aa => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:11.029618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:11.068367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:11.167793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f74c6e-8715-4ecb-b003-82400e0b5823 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:11.214647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff180919-7637-478b-baff-734915449cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:11.215268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff180919-7637-478b-baff-734915449cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:11.215542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff180919-7637-478b-baff-734915449cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:11.215814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff180919-7637-478b-baff-734915449cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:11.216562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff180919-7637-478b-baff-734915449cc4 None admin] RBAC: Adding query filter params (role.id=184d8fb55ab246ac8a358668dd43bb35, user.id=7e62753175b84fb481a1beca19cab0aa, scope.system=all) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:11.216865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff180919-7637-478b-baff-734915449cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:11.218298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff180919-7637-478b-baff-734915449cc4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:11.236313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 146/291] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Oct 27 13:11:11 2022] GET /identity/v3/role_assignments?user.id=7e62753175b84fb481a1beca19cab0aa&scope.system=all&role.id=184d8fb55ab246ac8a358668dd43bb35 => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:11.291379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:11.306821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bd828-ec9c-4127-92e9-0ca3d4ced9cd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:11.313561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:11.314208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:11.314476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:11.314737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:11.330989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:11.335924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:11.336848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:11.338285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 146/292] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:11.346136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff180919-7637-478b-baff-734915449cc4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:11.361702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff180919-7637-478b-baff-734915449cc4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:11.370471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:11.371104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:11.371376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:11.371646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:11.372362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:11.372663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:11.374272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:11.388834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 147/293] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Oct 27 13:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:11.394904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:11.412980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb4c6b-863d-434a-9ddd-ffe7f1ca5e2b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:11.417995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:11.418688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:11.419032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:11.419351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] PATH_INFO: `/v3/groups/789c45f366304767af6e06c75bd4ce84` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:11.431256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] RBAC: Authorizing `identity:get_group(group_id=789c45f366304767af6e06c75bd4ce84)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:11.433286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:11.435268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 147/294] 10.222.0.48 () {56 vars in 1201 bytes} [Thu Oct 27 13:11:11 2022] GET /identity/v3/groups/789c45f366304767af6e06c75bd4ce84 => generated 229 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:11.441569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:11.457550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b1df-b1f4-4051-9ecd-cd3c9aebe57f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:11.466127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927a517-c3d4-489d-9928-53f9be473791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:11.466766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927a517-c3d4-489d-9928-53f9be473791 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:11.467042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927a517-c3d4-489d-9928-53f9be473791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:11.467314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927a517-c3d4-489d-9928-53f9be473791 admin admin] PATH_INFO: `/v3/projects/f354da2936f649f78cdcb89056983d15` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:11.468563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927a517-c3d4-489d-9928-53f9be473791 admin admin] RBAC: Authorizing `identity:get_project(project_id=f354da2936f649f78cdcb89056983d15)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:11.470372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927a517-c3d4-489d-9928-53f9be473791 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:11.472362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 148/295] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Oct 27 13:11:11 2022] GET /identity/v3/projects/f354da2936f649f78cdcb89056983d15 => generated 344 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:11.478313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:11.493586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8a588-f836-41e9-9b55-5994d285f33d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:11.502335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302f8e6-212b-4ba9-bef4-ad59d991ca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:11.503009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302f8e6-212b-4ba9-bef4-ad59d991ca11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:11.503285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302f8e6-212b-4ba9-bef4-ad59d991ca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:11.503550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302f8e6-212b-4ba9-bef4-ad59d991ca11 admin admin] PATH_INFO: `/v3/projects/f354da2936f649f78cdcb89056983d15/groups/789c45f366304767af6e06c75bd4ce84/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:11.507254 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302f8e6-212b-4ba9-bef4-ad59d991ca11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f354da2936f649f78cdcb89056983d15, group_id=789c45f366304767af6e06c75bd4ce84, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:11.510472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302f8e6-212b-4ba9-bef4-ad59d991ca11 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:11.532316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:11.538438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 148/296] 10.222.0.48 () {58 vars in 1463 bytes} [Thu Oct 27 13:11:11 2022] PUT /identity/v3/projects/f354da2936f649f78cdcb89056983d15/groups/789c45f366304767af6e06c75bd4ce84/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:12.500956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03defeb5-7dac-44f4-9d83-67c2b62a5e15 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:12.501718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03defeb5-7dac-44f4-9d83-67c2b62a5e15 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:12.501718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03defeb5-7dac-44f4-9d83-67c2b62a5e15 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:12.503023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 149/297] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:12.508356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1e31-858c-4648-93ed-844769d942b8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:12.508896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1e31-858c-4648-93ed-844769d942b8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:12.509053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1e31-858c-4648-93ed-844769d942b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:12.516181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aede8-ea4f-4ce4-b9a2-ebba73279f04 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:12.516738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aede8-ea4f-4ce4-b9a2-ebba73279f04 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:12.517010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aede8-ea4f-4ce4-b9a2-ebba73279f04 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:12.518633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 150/298] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:12.524028 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1544a1-32f6-41a8-b8fc-e9c6585f0990 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:12.524333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1544a1-32f6-41a8-b8fc-e9c6585f0990 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:12.525222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1544a1-32f6-41a8-b8fc-e9c6585f0990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:12.550978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-df4b1e31-858c-4648-93ed-844769d942b8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:12.552169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-df4b1e31-858c-4648-93ed-844769d942b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:12.562241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2b1544a1-32f6-41a8-b8fc-e9c6585f0990 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:12.563298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2b1544a1-32f6-41a8-b8fc-e9c6585f0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:12.688964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:12.695817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 149/299] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:12.704727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 151/300] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:12.726942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a15a3-f87b-4f5c-a4df-b121c35fd13a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:12.727246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a15a3-f87b-4f5c-a4df-b121c35fd13a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:12.727485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a15a3-f87b-4f5c-a4df-b121c35fd13a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:12.728933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 150/301] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:12.735324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c7fc6-6198-4e01-93d0-2a628e835d1a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:12.735610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c7fc6-6198-4e01-93d0-2a628e835d1a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:12.735880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c7fc6-6198-4e01-93d0-2a628e835d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:12.761312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8a8c7fc6-6198-4e01-93d0-2a628e835d1a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:12.762858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8a8c7fc6-6198-4e01-93d0-2a628e835d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:12.836778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 152/302] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:13.046700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f50835-4e5f-4c66-9440-e9a4da940a1b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.047031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f50835-4e5f-4c66-9440-e9a4da940a1b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.047201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f50835-4e5f-4c66-9440-e9a4da940a1b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.048533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 151/303] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:13.053907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea1c9a-ab87-4711-b714-87deae607960 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.054191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea1c9a-ab87-4711-b714-87deae607960 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.054460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea1c9a-ab87-4711-b714-87deae607960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.056394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f50835-4e5f-4c66-9440-e9a4da940a1b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.105600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-13ea1c9a-ab87-4711-b714-87deae607960 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:13.106787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13ea1c9a-ab87-4711-b714-87deae607960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:13.206004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f50835-4e5f-4c66-9440-e9a4da940a1b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.206193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 153/304] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:13.207006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea1c9a-ab87-4711-b714-87deae607960 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.237608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:13.245793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.246528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.246799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.247062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.258084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:13.263816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:13.264718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:13.265507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:13.266840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 152/305] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:13.267393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.276305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:13.329964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea1c9a-ab87-4711-b714-87deae607960 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.373103 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.373734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.374005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.374271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.387518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:13.393681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:13.394587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 154/306] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:13.399339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.416328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f65e8-1a25-4a69-91f7-cc151f403e90 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.467169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.468878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.469407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.469861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.489342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:13.496294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:13.497175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:13.497944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:13.500291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 153/307] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:13.501563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.526611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e11c7-5f3c-4dcf-a045-4d8fa4863934 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.545382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:13.566441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.567302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.567729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.568613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.569306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] RBAC: Adding query filter params (name=member) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:13.569611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:13.571030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:13.577387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:13.581327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 155/308] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:13.582129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.644874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051aca-da39-471b-92be-c0d5fb77bbf7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.685921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.686682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.686959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.687229 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.688307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] RBAC: Adding query filter params (name=member) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:13.688597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:13.690383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:13.699476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 154/309] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:13.700375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.708419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441e444-fc7d-4856-bbbc-e28452c6bc5d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.756847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.758157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.758805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.759371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.760221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:13.760847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:13.763197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:13.774525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 156/310] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:13.775603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.829318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:13.839969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b03-314d-4f44-9172-34a0df139363 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.854927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:13.881950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.882581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.882856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.883123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:13.895081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cdad4-baf3-4e5f-92e9-35009e80806c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:13.896634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:13.902630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:13.903490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 155/311] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:13.904770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:13.948516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:13.949136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:13.949410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:13.949677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.031537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f5856-d0b7-4124-b7dc-5d0d4771cc19 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.043244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.045448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.079569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:14.080173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.081201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.081432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.101860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.140107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.182249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.182440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 157/312] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:14.183286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:14.184050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.308950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 156/313] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:14.310846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:14.316075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69681cd-b056-41a8-8fbc-47253410811c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.359148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:14.360017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.360349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.360627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.371747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.400369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.442916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6afb9-24f0-4e4c-ba72-d687c323b50a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.448668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.450426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.488598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:14.489208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.489488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.489756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.490580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:14.490883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.493109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.502626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 157/314] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:14.503450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:14.572838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 158/315] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:14.573858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:14.628983 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.629359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3ca06-e040-4c46-8b56-558a56f1e997 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.663546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:14.664161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.664464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.664727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.665922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.667331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.668964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 158/316] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:14.669710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:14.672088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.690717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c06e8-8c73-4b96-a1e9-6cbf757494a0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.724425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721979d-0f95-4fe6-9c31-6823a877fb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:14.725044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721979d-0f95-4fe6-9c31-6823a877fb1e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.725314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721979d-0f95-4fe6-9c31-6823a877fb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.725586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721979d-0f95-4fe6-9c31-6823a877fb1e None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.726666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721979d-0f95-4fe6-9c31-6823a877fb1e None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.728100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721979d-0f95-4fe6-9c31-6823a877fb1e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.730056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 159/317] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:14.731223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b4b17-49dd-4583-bff1-cc6b143047cf None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.731499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b4b17-49dd-4583-bff1-cc6b143047cf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.731760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b4b17-49dd-4583-bff1-cc6b143047cf None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.732972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 160/318] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:14.733647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b4b17-49dd-4583-bff1-cc6b143047cf None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:14.771283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd26df-cc97-4e34-9cb2-17ad8c71c6be None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.802660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:14.803263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.803538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.803804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.832567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b4b17-49dd-4583-bff1-cc6b143047cf None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.841489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.865276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:14.865882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:14.866161 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:14.866424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] PATH_INFO: `/v3/projects/dcc09680cf024f28a65aaca40b00c00f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:14.867550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc09680cf024f28a65aaca40b00c00f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.868857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:14.869433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.870283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:14.871014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 161/319] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/projects/dcc09680cf024f28a65aaca40b00c00f => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:14.871760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:14.872168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:14.967933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a3258-d252-4f30-9ea6-2dfad1aa2435 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:14.969958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 159/320] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:14.970500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.002631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407e09f-10bf-4c10-96de-210c358b4ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.003252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e09f-10bf-4c10-96de-210c358b4ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.003541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e09f-10bf-4c10-96de-210c358b4ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.003790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e09f-10bf-4c10-96de-210c358b4ed5 None admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.004874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e09f-10bf-4c10-96de-210c358b4ed5 None admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.006289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e09f-10bf-4c10-96de-210c358b4ed5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.007833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 162/321] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.008952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1e3d6-4815-457f-a23a-8880a0011da5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.009228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1e3d6-4815-457f-a23a-8880a0011da5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.009496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1e3d6-4815-457f-a23a-8880a0011da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.026865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2ad1e3d6-4815-457f-a23a-8880a0011da5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:15.028041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2ad1e3d6-4815-457f-a23a-8880a0011da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:15.065033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e225ec-fba2-4987-b6a1-3d82a7f83450 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.066498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:15.068720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:15.094798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 163/322] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:15.095647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1e3d6-4815-457f-a23a-8880a0011da5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.109558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.110189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.110486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.110751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.111877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.113300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.114870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 160/323] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Oct 27 13:11:14 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.115602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.207802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1e3d6-4815-457f-a23a-8880a0011da5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.215755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453556bc-f556-4e5e-82e3-ae4b67718c3e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.246339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.247451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.247962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.248415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.249541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.249904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.250164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.250427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.250689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] PATH_INFO: `/v3/projects/3a40440cb403480ca7194909d0e78c70` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.251802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] RBAC: Authorizing `identity:get_project(project_id=3a40440cb403480ca7194909d0e78c70)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.252667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.253226 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.256331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 161/324] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/projects/3a40440cb403480ca7194909d0e78c70 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.256331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.259425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 164/325] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.260131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.299352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:15.306410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:15.365981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8240dd-a494-4bcc-95ff-317d9b7d4833 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.367128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7c18-dec7-4808-aa10-b50fac3b457a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.400730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.401324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.401598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.401691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.401862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.402266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.402545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.402640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] RBAC: Adding query filter params (group.id=8bf7569f0b354ff099196a705c13d8be, role.id=111b31616dc54e43a9000ee72a3ac5b8, scope.project.id=3a40440cb403480ca7194909d0e78c70) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:15.402813 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.402906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.403518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] RBAC: Adding query filter params (group.id=8bf7569f0b354ff099196a705c13d8be, role.id=111b31616dc54e43a9000ee72a3ac5b8, scope.project.id=dcc09680cf024f28a65aaca40b00c00f) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:15.403816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.404284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.405227 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.409947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 162/326] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/role_assignments?group.id=8bf7569f0b354ff099196a705c13d8be&scope.project.id=3a40440cb403480ca7194909d0e78c70&role.id=111b31616dc54e43a9000ee72a3ac5b8 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.410706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.410868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 165/327] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/role_assignments?group.id=8bf7569f0b354ff099196a705c13d8be&scope.project.id=dcc09680cf024f28a65aaca40b00c00f&role.id=111b31616dc54e43a9000ee72a3ac5b8 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.411758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.541245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82752e-35a6-410f-b254-fd8892e405bd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.545073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ec028-11cf-457f-971e-5c963d6821a0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.545266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:15.552918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:15.585069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.586395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.587518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.587978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.588620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] PATH_INFO: `/v3/groups/8bf7569f0b354ff099196a705c13d8be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.588738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.588939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.589204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.589991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] RBAC: Adding query filter params (group.id=8bf7569f0b354ff099196a705c13d8be, role.id=fc711bb446524efbbb3ee062f6ff2702, scope.project.id=dcc09680cf024f28a65aaca40b00c00f) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:15.590224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.591059 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] RBAC: Authorizing `identity:get_group(group_id=8bf7569f0b354ff099196a705c13d8be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.591669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.592934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.595056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 163/328] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/groups/8bf7569f0b354ff099196a705c13d8be => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.596382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.599304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 166/329] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/role_assignments?group.id=8bf7569f0b354ff099196a705c13d8be&scope.project.id=dcc09680cf024f28a65aaca40b00c00f&role.id=fc711bb446524efbbb3ee062f6ff2702 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.605689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.740007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02048ea-7686-4ddc-b786-c2273d5f725c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.761155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ee7ce-cfd7-4776-9d68-d928f022b20e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.783225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.783871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.784144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.784439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.797276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:15.802936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:15.803808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:15.804635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 164/330] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:15.809618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:15.815568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.816447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.816722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.818082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.818082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] RBAC: Adding query filter params (group.id=8bf7569f0b354ff099196a705c13d8be, role.id=fc711bb446524efbbb3ee062f6ff2702, scope.project.id=3a40440cb403480ca7194909d0e78c70) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:15.818281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.819668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.828211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 167/331] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/role_assignments?group.id=8bf7569f0b354ff099196a705c13d8be&scope.project.id=3a40440cb403480ca7194909d0e78c70&role.id=fc711bb446524efbbb3ee062f6ff2702 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:15.849281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:15.928381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcebf72-2a3a-4e93-9614-94b33bf95a9d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:15.966769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:15.967398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:15.967678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:15.967948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:15.968667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:15.968969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:15.970346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:15.980029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 165/332] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:15.986079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:16.026485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:16.098147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ade48e-cd4f-4572-9b3f-b154a1455260 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:16.131162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:16.131767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:16.132039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:16.132330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:16.199637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:16.201110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:16.234982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:16.301788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 168/333] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:16.308092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:16.401463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:16.412113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770039da-13fc-4ae0-ab7a-da1503b65343 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:16.445079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:16.445703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:16.445978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:16.446835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] PATH_INFO: `/v3/projects/f354da2936f649f78cdcb89056983d15` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:16.447943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] RBAC: Authorizing `identity:get_project(project_id=f354da2936f649f78cdcb89056983d15)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:16.449386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:16.450992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 166/334] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:11:16 2022] GET /identity/v3/projects/f354da2936f649f78cdcb89056983d15 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:16.456700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:16.551021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3735-6c14-4ec7-b323-fce1f1cec8b1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:16.582826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc61ba-0301-40c7-83d6-87aa1e523abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:16.583454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc61ba-0301-40c7-83d6-87aa1e523abc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:16.583728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc61ba-0301-40c7-83d6-87aa1e523abc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:16.583997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc61ba-0301-40c7-83d6-87aa1e523abc None admin] PATH_INFO: `/v3/groups/789c45f366304767af6e06c75bd4ce84` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:16.585107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc61ba-0301-40c7-83d6-87aa1e523abc None admin] RBAC: Authorizing `identity:get_group(group_id=789c45f366304767af6e06c75bd4ce84)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:16.586514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc61ba-0301-40c7-83d6-87aa1e523abc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:16.588037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 169/335] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Oct 27 13:11:16 2022] GET /identity/v3/groups/789c45f366304767af6e06c75bd4ce84 => generated 229 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:16.593864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:16.689897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cdf76-b736-4de4-8983-359e3e4d91c4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:16.720811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50a35e-3d87-4411-b757-f9f47d913e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:16.721396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50a35e-3d87-4411-b757-f9f47d913e94 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:16.721671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50a35e-3d87-4411-b757-f9f47d913e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:16.721937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50a35e-3d87-4411-b757-f9f47d913e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:16.722642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50a35e-3d87-4411-b757-f9f47d913e94 None admin] RBAC: Adding query filter params (group.id=789c45f366304767af6e06c75bd4ce84, role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=f354da2936f649f78cdcb89056983d15) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:16.722940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50a35e-3d87-4411-b757-f9f47d913e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:16.724376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50a35e-3d87-4411-b757-f9f47d913e94 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:16.730404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 167/336] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Oct 27 13:11:16 2022] GET /identity/v3/role_assignments?group.id=789c45f366304767af6e06c75bd4ce84&scope.project.id=f354da2936f649f78cdcb89056983d15&role.id=33f912ad05b44e36a696ef6aa86733be => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:19.252665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d2eb7-faf1-4164-afe9-269e11199af2 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:19.253230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d2eb7-faf1-4164-afe9-269e11199af2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:19.253230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d2eb7-faf1-4164-afe9-269e11199af2 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:19.254938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 170/337] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:19.263765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfe366-4c4b-4980-9d25-65691f85774d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:19.264621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfe366-4c4b-4980-9d25-65691f85774d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:19.265090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfe366-4c4b-4980-9d25-65691f85774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:19.289193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-09bfe366-4c4b-4980-9d25-65691f85774d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:19.291234 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-09bfe366-4c4b-4980-9d25-65691f85774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:19.317062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:19.385547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 168/338] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:19.638460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68d69a-7258-47aa-bc32-8e5638469a2b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:19.639104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68d69a-7258-47aa-bc32-8e5638469a2b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:19.639583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68d69a-7258-47aa-bc32-8e5638469a2b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:19.641398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 171/339] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:19.647088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c7615-4f87-4703-83f9-c8476fba97dc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:19.647602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c7615-4f87-4703-83f9-c8476fba97dc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:19.648097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c7615-4f87-4703-83f9-c8476fba97dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:19.677881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-289c7615-4f87-4703-83f9-c8476fba97dc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:19.679377 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-289c7615-4f87-4703-83f9-c8476fba97dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:19.721275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68d69a-7258-47aa-bc32-8e5638469a2b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:19.724196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:19.780818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 169/340] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:19.837812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9573ff-b23d-4360-96fc-5c706aec864e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:19.838440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9573ff-b23d-4360-96fc-5c706aec864e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:19.838915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9573ff-b23d-4360-96fc-5c706aec864e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:19.840503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 170/341] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:19.845783 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87fde4-6093-4fe4-a4de-899fb8dc2f49 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:19.846304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87fde4-6093-4fe4-a4de-899fb8dc2f49 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:19.846769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87fde4-6093-4fe4-a4de-899fb8dc2f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:19.869777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-af87fde4-6093-4fe4-a4de-899fb8dc2f49 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:19.871360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-af87fde4-6093-4fe4-a4de-899fb8dc2f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:19.903241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68d69a-7258-47aa-bc32-8e5638469a2b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:19.972286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c337d-ef7f-4a70-848d-680a374b96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:19.973292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c337d-ef7f-4a70-848d-680a374b96ec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:19.973786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c337d-ef7f-4a70-848d-680a374b96ec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:19.974287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c337d-ef7f-4a70-848d-680a374b96ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:19.985583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 171/342] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:19.988906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c337d-ef7f-4a70-848d-680a374b96ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:19.990582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c337d-ef7f-4a70-848d-680a374b96ec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-142c337d-ef7f-4a70-848d-680a374b96ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:20.009336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:20.010312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:20.011193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:20.011193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.011193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:20.017956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 172/343] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:20.024176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87fde4-6093-4fe4-a4de-899fb8dc2f49 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.046700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f068-7985-4a7c-a9ca-25ebc3fcee0a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.047401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f068-7985-4a7c-a9ca-25ebc3fcee0a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.047899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f068-7985-4a7c-a9ca-25ebc3fcee0a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.049474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 173/344] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:20.052256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:20.055393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b88f-ea74-4de8-a51c-b0a173cfdb2e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.056034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b88f-ea74-4de8-a51c-b0a173cfdb2e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.056579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b88f-ea74-4de8-a51c-b0a173cfdb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.076152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0966b88f-ea74-4de8-a51c-b0a173cfdb2e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:20.078235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0966b88f-ea74-4de8-a51c-b0a173cfdb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:20.136840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:20.155768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 174/345] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:20.164252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966b88f-ea74-4de8-a51c-b0a173cfdb2e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.189966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87fde4-6093-4fe4-a4de-899fb8dc2f49 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.256976 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.257577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.257874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.258121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.258785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:20.259072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.260443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.269153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966b88f-ea74-4de8-a51c-b0a173cfdb2e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.284991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 172/346] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:20.297487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.318430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.319062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.319501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.319947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.331400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.332916 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:20.349629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:20.350720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:20.351779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:20.351779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.351779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:20.351779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 175/347] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:20.351779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.394816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:20.408614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:20.436076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab61a48-54c9-455d-a0f9-5eca8bbc3dbf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.471586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.473013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.473458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.473882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.474726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.475120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7aa28-b63f-470c-8143-13ad1079ffdf None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.476362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.504334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 173/348] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:11:20 2022] POST /identity/v3/users => generated 246 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:20.505510 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.522845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.523802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.524323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.524763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.537434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.539156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:20.556223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:20.557375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:20.558417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:20.558417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.558417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:20.559862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 176/349] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:20.566143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.639273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744c2dc-80a1-4ca2-8e7b-f7244ff01d8a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.675849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:20.684498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.685319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.685700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.686074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.686837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:20.687306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.687765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:20.690636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.702138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be3b1e-8832-47f4-a50d-59ede3b49cb3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.709514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 174/350] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:20.709514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.746091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.746444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.746714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.746989 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.761978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.763791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:20.785116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:20.786051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:20.786925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:20.786925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:20.786925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:20.787509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 177/351] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:20.788104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.839519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaf36b-f8fe-4cb2-9a0c-f6fb1211a12d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.885215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.885833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.886382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.886662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.887442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:20.887761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.889765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.896918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f62c-1038-4e89-935f-1ea6cf195bec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:20.905480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 175/352] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:20.906151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.934392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:20.935011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:20.935303 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:20.935578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:20.936256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:20.937849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:20.950786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:20.963202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 178/353] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:11:20 2022] POST /identity/v3/users => generated 248 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:20.964341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:20.978329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:21.086035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c5222-9552-467f-9591-a69cf0941f3e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:21.089906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614f0c6-f1b7-4973-9be6-160c922f854a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:21.128541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c27220-b03f-4ec5-9e49-5baef45260b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:21.129277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c27220-b03f-4ec5-9e49-5baef45260b2 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:21.129643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c27220-b03f-4ec5-9e49-5baef45260b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:21.129987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c27220-b03f-4ec5-9e49-5baef45260b2 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:21.130779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c27220-b03f-4ec5-9e49-5baef45260b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:21.132391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c27220-b03f-4ec5-9e49-5baef45260b2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:21.132715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c15e4d-3c36-485c-ba98-5ff0841d37be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:21.133327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15e4d-3c36-485c-ba98-5ff0841d37be None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:21.133602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15e4d-3c36-485c-ba98-5ff0841d37be None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:21.134451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15e4d-3c36-485c-ba98-5ff0841d37be None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:21.135094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c15e4d-3c36-485c-ba98-5ff0841d37be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:21.135434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c15e4d-3c36-485c-ba98-5ff0841d37be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:21.136978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c15e4d-3c36-485c-ba98-5ff0841d37be None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:21.151754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 176/354] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:21.153785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6d0d-2f5e-4f8b-a85c-ac46f89d546a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:21.154064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6d0d-2f5e-4f8b-a85c-ac46f89d546a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:21.154329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff6d0d-2f5e-4f8b-a85c-ac46f89d546a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:21.156346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 177/355] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:21.161292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff6d0d-2f5e-4f8b-a85c-ac46f89d546a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:21.163910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 179/356] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:11:20 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:21.165730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae064e6-b638-43d4-9c71-eac3b8686e7f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:21.166083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae064e6-b638-43d4-9c71-eac3b8686e7f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:21.166432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae064e6-b638-43d4-9c71-eac3b8686e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:21.188764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9ae064e6-b638-43d4-9c71-eac3b8686e7f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:21.190261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9ae064e6-b638-43d4-9c71-eac3b8686e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:21.255497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:21.294055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 180/357] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:21.322293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff6d0d-2f5e-4f8b-a85c-ac46f89d546a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:21.336139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:21.368939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:21.369863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:21.370291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:21.370696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:21.371583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:21.373250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:21.403769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 178/358] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:11:21 2022] POST /identity/v3/users => generated 249 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:21.602234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae064e6-b638-43d4-9c71-eac3b8686e7f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:21.730299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae064e6-b638-43d4-9c71-eac3b8686e7f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:21.778243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:21.778854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:21.779134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:21.779399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:21.780070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:21.781448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:21.797670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.sql.core [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 27 13:11:21.797670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 27 13:11:21.797670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [parameters: {'id': '8c90c3708ad540e890bf920bf98969ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 27 13:11:21.797670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81442) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result.read() Oct 27 13:11:21.804050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return ref.to_dict() Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:11:21.804930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application yield resource Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise self.value Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application session.commit() Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 13:11:21.806451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.session.flush() Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application compat.raise_( Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 13:11:21.807316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:11:21.808200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application result.read() Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:11:21.809031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [parameters: {'id': '8c90c3708ad540e890bf920bf98969ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:21.809810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 27 13:11:21.810676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:21.811506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 181/359] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Oct 27 13:11:21 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:21.815021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:22.007982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08491184-4f1b-4937-97cc-00e69ac36359 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:22.052045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594c125-8c7f-4e49-bb92-794bc9e12392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:22.053067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594c125-8c7f-4e49-bb92-794bc9e12392 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:22.053565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594c125-8c7f-4e49-bb92-794bc9e12392 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:22.054065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594c125-8c7f-4e49-bb92-794bc9e12392 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:22.073795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594c125-8c7f-4e49-bb92-794bc9e12392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-1594c125-8c7f-4e49-bb92-794bc9e12392 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:22.079190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:22.080070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:22.080893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 179/360] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:11:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:22.086217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:22.151491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:22.227990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383b373-0fc5-4ae2-8e03-d61afa72a871 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:22.280470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f00de-081b-4501-a8f3-fb2e4c01c7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:22.280996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f00de-081b-4501-a8f3-fb2e4c01c7ea None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:22.281270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f00de-081b-4501-a8f3-fb2e4c01c7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:22.281539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f00de-081b-4501-a8f3-fb2e4c01c7ea None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:22.282198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f00de-081b-4501-a8f3-fb2e4c01c7ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:22.282494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f00de-081b-4501-a8f3-fb2e4c01c7ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:22.283851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f00de-081b-4501-a8f3-fb2e4c01c7ea None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:22.295811 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 182/361] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:23.151751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac30ac-7e2c-404e-8406-e3ec1625e891 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:23.152292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac30ac-7e2c-404e-8406-e3ec1625e891 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:23.152292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac30ac-7e2c-404e-8406-e3ec1625e891 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:23.153922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 180/362] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:23.159314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e30c7-9fee-4288-b0fc-6cbc07c892e3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:23.159603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e30c7-9fee-4288-b0fc-6cbc07c892e3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:23.159876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e30c7-9fee-4288-b0fc-6cbc07c892e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:23.193552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-376e30c7-9fee-4288-b0fc-6cbc07c892e3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:23.194724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-376e30c7-9fee-4288-b0fc-6cbc07c892e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:23.284630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 183/363] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:23.567572 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f20bd-3432-4852-bbcc-0585337d61e4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:23.567753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f20bd-3432-4852-bbcc-0585337d61e4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:23.568034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f20bd-3432-4852-bbcc-0585337d61e4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:23.569624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 181/364] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:23.575700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206d42-d2df-417d-b9f6-101c398ed979 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:23.576139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206d42-d2df-417d-b9f6-101c398ed979 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:23.576213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206d42-d2df-417d-b9f6-101c398ed979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:23.608387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d5206d42-d2df-417d-b9f6-101c398ed979 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:23.609551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d5206d42-d2df-417d-b9f6-101c398ed979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:23.657417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:23.700530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f20bd-3432-4852-bbcc-0585337d61e4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:23.720481 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 184/365] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:23.735144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:23.816809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f20bd-3432-4852-bbcc-0585337d61e4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:23.852035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:23.852970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:23.853413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:23.853897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:23.864376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:23.870060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:23.870918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:23.871720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:23.874047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 182/366] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:23.880888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206d42-d2df-417d-b9f6-101c398ed979 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.004493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206d42-d2df-417d-b9f6-101c398ed979 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.057612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.058484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.058896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.059340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.060127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:24.060594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:24.062080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:24.073438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 185/367] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:24.078920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.116660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.164387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:24.202208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:24.222201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3af8d-bfb0-4588-9add-22317b335bf8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.258600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e1c4e-d490-47f7-9430-ff42620ffd6b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.262589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69818771-5712-4bd0-8a7b-0f286a1079c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.263447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818771-5712-4bd0-8a7b-0f286a1079c9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.263935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818771-5712-4bd0-8a7b-0f286a1079c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.264428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818771-5712-4bd0-8a7b-0f286a1079c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.275351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69818771-5712-4bd0-8a7b-0f286a1079c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:24.277056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69818771-5712-4bd0-8a7b-0f286a1079c9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-69818771-5712-4bd0-8a7b-0f286a1079c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:24.292269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:24.293154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:24.293990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:24.293990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:24.293990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:24.296275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 183/368] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:24.297685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5979-a19e-4951-84fb-2ac6dce75fac None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.298153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5979-a19e-4951-84fb-2ac6dce75fac None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.298626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5979-a19e-4951-84fb-2ac6dce75fac None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.300025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 184/369] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:24.301401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def5979-a19e-4951-84fb-2ac6dce75fac None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.306928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33869348-39ae-4ae0-9477-1f730c9a6aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.307556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869348-39ae-4ae0-9477-1f730c9a6aec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.307828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869348-39ae-4ae0-9477-1f730c9a6aec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.308085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869348-39ae-4ae0-9477-1f730c9a6aec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.323420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33869348-39ae-4ae0-9477-1f730c9a6aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-33869348-39ae-4ae0-9477-1f730c9a6aec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:24.329977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:24.330953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:24.331726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 186/370] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:24.333505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31487-79c2-4d11-b086-3ccc3dc744d5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.333838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31487-79c2-4d11-b086-3ccc3dc744d5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.334113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31487-79c2-4d11-b086-3ccc3dc744d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.357970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-05e31487-79c2-4d11-b086-3ccc3dc744d5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:24.359898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-05e31487-79c2-4d11-b086-3ccc3dc744d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:24.410832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def5979-a19e-4951-84fb-2ac6dce75fac None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.442089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 187/371] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:24.443362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bc068-3534-4523-97a8-e6f98d8f817f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.443643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bc068-3534-4523-97a8-e6f98d8f817f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.443911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bc068-3534-4523-97a8-e6f98d8f817f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.445336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 188/372] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:24.449717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bc068-3534-4523-97a8-e6f98d8f817f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.455062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024848cf-a93b-446c-8ecd-3a495a05b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.455710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024848cf-a93b-446c-8ecd-3a495a05b8fa None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.456001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024848cf-a93b-446c-8ecd-3a495a05b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.456273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024848cf-a93b-446c-8ecd-3a495a05b8fa None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.456905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024848cf-a93b-446c-8ecd-3a495a05b8fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:24.457255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024848cf-a93b-446c-8ecd-3a495a05b8fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:24.458667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024848cf-a93b-446c-8ecd-3a495a05b8fa None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:24.466533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:24.473051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 185/373] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:24.474443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91bd36-0b4b-41c5-a75b-09171e2c008d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.474820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91bd36-0b4b-41c5-a75b-09171e2c008d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.475196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91bd36-0b4b-41c5-a75b-09171e2c008d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.484543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:24.495250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6f91bd36-0b4b-41c5-a75b-09171e2c008d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:24.496801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6f91bd36-0b4b-41c5-a75b-09171e2c008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:24.565764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 186/374] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:24.566955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91bd36-0b4b-41c5-a75b-09171e2c008d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.585870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bc068-3534-4523-97a8-e6f98d8f817f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.662462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.662985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.663255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.663519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.668823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:24.669137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:24.672070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:24.693151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 189/375] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:24.695644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.738785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91bd36-0b4b-41c5-a75b-09171e2c008d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.779086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:24.793851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.795042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.795755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.796664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.806707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:24.808427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:24.816957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:24.827519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:24.828824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:24.829685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:24.831003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 187/376] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:24.832097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.842929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68267c85-085e-45df-b57e-b7509058cfe6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.878789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677110e-6c60-410c-a909-dee157941742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.879419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677110e-6c60-410c-a909-dee157941742 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.879694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677110e-6c60-410c-a909-dee157941742 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.879961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677110e-6c60-410c-a909-dee157941742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:24.892647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677110e-6c60-410c-a909-dee157941742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:24.894122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677110e-6c60-410c-a909-dee157941742 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d677110e-6c60-410c-a909-dee157941742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:24.910058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:24.911051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:24.911879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:24.911879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:24.911879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:24.911879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 190/377] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:24.912089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677110e-6c60-410c-a909-dee157941742 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:24.957975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b5515-d65e-4370-b954-ba1987117ee5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:24.990041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:24.990672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:24.990947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:24.991218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.001123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:25.006328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:25.007186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 188/378] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:25.009443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.022912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677110e-6c60-410c-a909-dee157941742 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.058552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.059147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.059420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.059623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.059716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.060415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:25.060710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.062183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.064161 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.099249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 191/379] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Oct 27 13:11:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:25.100060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.112619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562981b7-8664-4b7b-9dc7-dacfbe9f00f1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.146143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.146763 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.147036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.147304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.157052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:25.162132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:25.163041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:25.163814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 189/380] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:25.164504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.201817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047400fd-76ae-43fc-bece-802a472b9380 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.234829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b34dc-4342-4cc5-9e9d-995dcff23b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.235366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b34dc-4342-4cc5-9e9d-995dcff23b34 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.235642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b34dc-4342-4cc5-9e9d-995dcff23b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.235916 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b34dc-4342-4cc5-9e9d-995dcff23b34 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.236644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b34dc-4342-4cc5-9e9d-995dcff23b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:25.236907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b34dc-4342-4cc5-9e9d-995dcff23b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.238283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b34dc-4342-4cc5-9e9d-995dcff23b34 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.249810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 192/381] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:25.250957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e81f98-ae4c-47f4-96a6-c2cb5f9f0fdc None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.251235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e81f98-ae4c-47f4-96a6-c2cb5f9f0fdc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.251501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e81f98-ae4c-47f4-96a6-c2cb5f9f0fdc None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.252722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 193/382] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:25.253396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e81f98-ae4c-47f4-96a6-c2cb5f9f0fdc None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.263593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f5654-66d1-43a2-a5dd-dc3c55b591be None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.272874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.297837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.298426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.298700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.298965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.299610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:25.299905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.301286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.307158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.309705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 190/383] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:25.310532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.354668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e81f98-ae4c-47f4-96a6-c2cb5f9f0fdc None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.391347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.392255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.392526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.392795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.402566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.403998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.407639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd07621-d8fb-4833-8429-feed60673db1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:25.418075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:25.419115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:25.424674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:25.424674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:25.424674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:25.424674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 194/384] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:25.424674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.447855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41b75f-3d9e-4a6a-a38f-f5b699453906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.448484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41b75f-3d9e-4a6a-a38f-f5b699453906 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.448759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41b75f-3d9e-4a6a-a38f-f5b699453906 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.449035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41b75f-3d9e-4a6a-a38f-f5b699453906 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.449687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41b75f-3d9e-4a6a-a38f-f5b699453906 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:25.449999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41b75f-3d9e-4a6a-a38f-f5b699453906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.451342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41b75f-3d9e-4a6a-a38f-f5b699453906 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.459213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 191/385] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:25.460351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8133f6-f910-4437-9dd3-78811cf33353 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.460631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8133f6-f910-4437-9dd3-78811cf33353 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.460906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8133f6-f910-4437-9dd3-78811cf33353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.480107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ac8133f6-f910-4437-9dd3-78811cf33353 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:25.481318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac8133f6-f910-4437-9dd3-78811cf33353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:25.506665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.520269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1d3f-e4b5-4331-96b5-06ceebaa12b6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.522502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.546658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 192/386] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:25.547574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8133f6-f910-4437-9dd3-78811cf33353 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.567208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.567819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.568092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.568386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.575738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.577182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:25.591391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:25.592973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 195/387] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:25.593841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.658569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8133f6-f910-4437-9dd3-78811cf33353 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.695476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.696163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.696463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.696726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.697807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2e17f-56ee-4a00-96ac-b3083f9a3c3d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.706597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.708272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:25.722436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:25.723293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:25.724131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:25.724131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:25.724131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:25.724131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 193/388] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:25.724867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.740201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.740835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.741115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.741384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.742016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:25.742320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.742985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.744301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.756759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 196/389] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:25.757496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.783012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:25.837156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee9e8a-1e2f-4257-b063-f6e12fb5f079 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.860285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126c432-eb12-41c3-b573-b0b8bd75dd2c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:25.869806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.870488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.870761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.871025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.881199 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.882636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.893826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:25.894431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:25.894708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:25.894978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:25.895688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:25.895987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:25.897430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:25.897583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:25.898547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:25.899922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:25.899922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:25.899922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:25.899922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 194/390] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:25.900464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.920007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 197/391] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:25.920987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:25.966889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.001140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3efd0-be1e-4fe8-b235-35fdc75375e3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.001615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.019800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa902e1-8421-418d-a675-b0233b831e81 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.034099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.034700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.034978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.035250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.035889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:26.036194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.037597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.053926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 195/392] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:26.053926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.054251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.054251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.054251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.054495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.061029 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.062458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:26.074176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:26.075138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 198/393] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:26.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.150511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05565351-e7dd-4baa-8e6a-fe46f9358444 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.173785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa6457-d27e-41b8-8eaf-397ec499f86b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.176190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.185117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.185736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.186011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.186278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.195941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.197931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.208043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.208668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.208948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.209218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.209847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:26.210149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.211488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:26.212676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:26.213617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:26.214460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:26.214460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:26.214460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:26.214460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 196/394] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:26.214692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.229496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 199/395] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:26.230298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.232874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.315978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73735e2c-ee91-4ae9-a091-15d5fa98da36 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.328025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0577c8d-396f-4142-8b93-7d9ec5ebf51a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.350446 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.351061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.351331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.351601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.362043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.362252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.362905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.363179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.363445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.363630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.370460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.371883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:26.377281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:26.378198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:26.379318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:26.379318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:26.379318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:26.379318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 197/396] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:26.379728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:26.383121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:26.384659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 13:11:26.385522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:26.388380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 200/397] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:26.389408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.413719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.450068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.482737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868bc358-6f83-45ee-94e4-f6e473aa65f4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.488638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8d52a-a895-4ce4-934a-5a7d81299fc3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.515250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.515845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.516122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.516436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.517375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:26.517677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.519082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.521232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.521828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.522104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.522373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.523000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:26.523299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.525256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.536461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 198/398] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:26.541604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 201/399] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:26.541738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.541825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.629189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.639373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cf451-2f45-4570-b3da-ea1ba5572c63 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.639528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4f3fb-5750-4207-b476-7cdf899a28cb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.661823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.672732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.673068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.673358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.673637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.673728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.673908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.673993 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.674197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.674821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:26.675113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.676495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.681321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.682740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.687794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 202/400] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:26.688607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:26.694403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:26.695460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 199/401] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:26.701531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.788532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b627e-17ce-4ae2-8c37-67a79e806755 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.793450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bf76b-512b-4d75-a9fc-0d3f6274f182 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.822913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.823517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.823794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.824091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.824802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:26.825108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.826516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.826889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:26.827478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:26.827761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:26.828034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:26.828706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:26.830123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:26.843110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 203/402] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:26.844030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.850465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 200/403] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:11:26 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:26.851422 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:26.860747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.898854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:26.964914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e43e5-a4ad-49fa-9b0f-ed234524dc1b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:26.965132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1287fde-a518-4c33-a440-f9304b79cd3e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.003639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.004047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.004311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.004592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.004752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.004844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.005023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.005297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/d1d142bc3a9e4b3badefe713c7405329/roles/eb326c809f1543ce8be84f07a0a1cdab` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.012946 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.014436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.022299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=d1d142bc3a9e4b3badefe713c7405329, role_id=eb326c809f1543ce8be84f07a0a1cdab)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.023787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:27.027459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:27.028410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 201/404] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:27.029726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.044216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 204/405] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:26 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/d1d142bc3a9e4b3badefe713c7405329/roles/eb326c809f1543ce8be84f07a0a1cdab => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:27.045337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.130869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:27.168324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:27.173739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8f8d3-f617-4e0b-9d9b-a088cc8b488a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.182409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1a6c7-3243-4c7f-a185-120828904b00 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.225440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.226342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.226822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.227231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.227931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.228558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.228839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.229110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.229313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:27.229760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.231178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.242516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.244840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:27.261456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:27.262564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:27.263603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:27.263603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:27.263603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:27.263603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 205/406] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:27.263861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.278012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 202/407] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:27.279091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.379402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988ed5c-182b-409f-ae01-13fa7bd2357b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.385648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b560-3c76-471d-98ab-a95873e5ebda None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.423200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.423656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.423979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.424284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.424489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.424590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.424818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.425018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.425503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:27.425802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.425899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:27.426287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.427229 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.427843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.430049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:27.441056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 203/408] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:27.442036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.448857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 206/409] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:27.449707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.457127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:27.564172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47c5ff-b4df-48ea-bded-767ff47180d8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.578721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba7fe5-0285-4436-bcff-0dfd712b1754 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.608395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.609042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.609321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.609592 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.622400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.624554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.629804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.630736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.631208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.631914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:27.638879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:27.640176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:27.641043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 207/410] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:27.641488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.650174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.651871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:27.670497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:27.671415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:27.672278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:27.672278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:27.672278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:27.674707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 204/411] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:27.675808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.714897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:27.731455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:27.761357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee2a35-5113-4114-a114-e18c42803ee9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.807037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea6fea-2788-46e9-aaf7-03b1ca0cf721 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:27.809177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.810929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.811353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.811780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/80a57dd6e99c4cac9e6bcd1413afd5c0/roles/eb326c809f1543ce8be84f07a0a1cdab` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.833521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=80a57dd6e99c4cac9e6bcd1413afd5c0, role_id=eb326c809f1543ce8be84f07a0a1cdab)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.835839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.845261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:27.846111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:27.846399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:27.846791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:27.847502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:27.847969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:27.849664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:27.856139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 208/412] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:27 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/80a57dd6e99c4cac9e6bcd1413afd5c0/roles/eb326c809f1543ce8be84f07a0a1cdab => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:27.857218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.865085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 205/413] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:27.875981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:27.985896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5294-b8cb-4698-8074-19fd8d452819 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:28.003842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:28.046028 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:28.046860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:28.047336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:28.047731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:28.048517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:28.048948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:28.050530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:28.058257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:28.067077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74908f4-0ce9-45a2-885a-4ca1a4df3a50 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:28.069414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 209/414] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:28.075248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:28.116409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:28.117508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:28.117979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:28.119155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/b7cba7b6049e41488f5b2f7c61a0500a/roles/eb326c809f1543ce8be84f07a0a1cdab` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:28.136756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=b7cba7b6049e41488f5b2f7c61a0500a, role_id=eb326c809f1543ce8be84f07a0a1cdab)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:28.139082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:28.158137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 206/415] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:27 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/b7cba7b6049e41488f5b2f7c61a0500a/roles/eb326c809f1543ce8be84f07a0a1cdab => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:28.199467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c6523-1eea-4619-a8ff-71fe33722e94 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:28.237487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:28.238383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:28.238800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:28.239216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:28.247001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:28.248662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:28.261623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:28.262694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:28.263719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:28.265455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 210/416] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:28.275914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:28.406516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70e703-d8e3-450f-89bd-93e19b0f94b4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:28.443787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf1b51-2cf7-4677-8f8b-22cb1b687d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:28.444706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf1b51-2cf7-4677-8f8b-22cb1b687d3e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:28.445120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf1b51-2cf7-4677-8f8b-22cb1b687d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:28.445524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf1b51-2cf7-4677-8f8b-22cb1b687d3e None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:28.446933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf1b51-2cf7-4677-8f8b-22cb1b687d3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:28.447385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf1b51-2cf7-4677-8f8b-22cb1b687d3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:28.448972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf1b51-2cf7-4677-8f8b-22cb1b687d3e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:28.462667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 207/417] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:28.469243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:28.510323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:28.591676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87325091-a4b9-4bc3-8e9a-7583d2cc2967 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:28.643042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce025c-ca64-4b8e-8687-d65b7ee72e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:28.644392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce025c-ca64-4b8e-8687-d65b7ee72e0c None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:28.644807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce025c-ca64-4b8e-8687-d65b7ee72e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:28.645216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce025c-ca64-4b8e-8687-d65b7ee72e0c None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/2d4ce3e02e5945ffbbde389398b146c4/roles/eb326c809f1543ce8be84f07a0a1cdab` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:28.666482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce025c-ca64-4b8e-8687-d65b7ee72e0c None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=2d4ce3e02e5945ffbbde389398b146c4, role_id=eb326c809f1543ce8be84f07a0a1cdab)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:28.668136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce025c-ca64-4b8e-8687-d65b7ee72e0c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:28.685474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 211/418] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:28 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/2d4ce3e02e5945ffbbde389398b146c4/roles/eb326c809f1543ce8be84f07a0a1cdab => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:29.480188 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ecd9e-ebef-4145-8e80-2ba50ba37958 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:29.480745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ecd9e-ebef-4145-8e80-2ba50ba37958 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:29.481006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ecd9e-ebef-4145-8e80-2ba50ba37958 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:29.482709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 208/419] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:29.488139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65da810-44f1-4e74-bec4-4561bf0c0816 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:29.488601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65da810-44f1-4e74-bec4-4561bf0c0816 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:29.488882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65da810-44f1-4e74-bec4-4561bf0c0816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:29.500558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb96fd0-72af-4270-a858-6660608b987e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:29.500875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb96fd0-72af-4270-a858-6660608b987e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:29.501100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb96fd0-72af-4270-a858-6660608b987e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:29.502740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 209/420] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:29.508263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abd25d-4832-4f39-a692-2e9c6fb47fe7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:29.509429 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abd25d-4832-4f39-a692-2e9c6fb47fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:29.509714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abd25d-4832-4f39-a692-2e9c6fb47fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:29.518981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b65da810-44f1-4e74-bec4-4561bf0c0816 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:29.520166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b65da810-44f1-4e74-bec4-4561bf0c0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:29.534584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:29.542790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c9abd25d-4832-4f39-a692-2e9c6fb47fe7 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:29.543977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c9abd25d-4832-4f39-a692-2e9c6fb47fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:29.631619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 212/421] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:29.658464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 210/422] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:29.947200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65da810-44f1-4e74-bec4-4561bf0c0816 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:29.961779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:30.009761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abd25d-4832-4f39-a692-2e9c6fb47fe7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:30.060780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65da810-44f1-4e74-bec4-4561bf0c0816 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.099437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:30.100584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.101120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.101596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.123386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:30.128482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:30.129350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:30.130136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:30.139308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 213/423] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:30.149923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:30.229254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:30.256538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abd25d-4832-4f39-a692-2e9c6fb47fe7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.285245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b44b81-5f54-4e5a-8c0f-46fe090618ee None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.324412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:30.325691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.326172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.326650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.327523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:30.328022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:30.329990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:30.340014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 214/424] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:30.352042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:30.353288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.353565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.353831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.354561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:30.364561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:30.370080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:30.375166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:30.376074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:30.377830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:30.378963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 211/425] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:30.382260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:30.501128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5255d-a07d-4c77-b221-87bd0b69acb0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.545881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026a7c7-3f4f-4fa7-ba82-79c6162e14f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:30.546554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026a7c7-3f4f-4fa7-ba82-79c6162e14f6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.546833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026a7c7-3f4f-4fa7-ba82-79c6162e14f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.547099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026a7c7-3f4f-4fa7-ba82-79c6162e14f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.549598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:30.562317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026a7c7-3f4f-4fa7-ba82-79c6162e14f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:30.564854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026a7c7-3f4f-4fa7-ba82-79c6162e14f6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:30.572317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81397dfe-d3e1-40ca-9b25-5b4ef255cea7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-2026a7c7-3f4f-4fa7-ba82-79c6162e14f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:30.581630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:30.582638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:30.583685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:30.583685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:30.583685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:30.589754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 215/426] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:30.589754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd019731-bc6a-46ee-ad6c-9b18236d647a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.589754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd019731-bc6a-46ee-ad6c-9b18236d647a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.589754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd019731-bc6a-46ee-ad6c-9b18236d647a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.593744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 216/427] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:30.597183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd019731-bc6a-46ee-ad6c-9b18236d647a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:30.623713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4f8b-dcac-4f2d-996c-07dabfd806ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:30.624696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4f8b-dcac-4f2d-996c-07dabfd806ad None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.625198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4f8b-dcac-4f2d-996c-07dabfd806ad None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.625677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4f8b-dcac-4f2d-996c-07dabfd806ad None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.626557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4f8b-dcac-4f2d-996c-07dabfd806ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:30.627159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4f8b-dcac-4f2d-996c-07dabfd806ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:30.628882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4f8b-dcac-4f2d-996c-07dabfd806ad None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:30.641164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 212/428] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:30.642982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e4990-9bb8-461d-b129-17cecfa86adb None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.643490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e4990-9bb8-461d-b129-17cecfa86adb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.643975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e4990-9bb8-461d-b129-17cecfa86adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.668349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2a8e4990-9bb8-461d-b129-17cecfa86adb None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:30.670608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2a8e4990-9bb8-461d-b129-17cecfa86adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:30.711680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd019731-bc6a-46ee-ad6c-9b18236d647a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.736574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:30.748059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:30.748963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.749451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.749926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.750858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:30.751429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:30.753297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:30.760252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 213/429] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:30.760252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e4990-9bb8-461d-b129-17cecfa86adb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:30.780651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 217/430] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:30.786309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:30.871059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:30.935160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19505a8-1f59-46c3-ae39-7c4238b1b48d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.962523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e4990-9bb8-461d-b129-17cecfa86adb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:30.972362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ac147-b71e-4d05-b884-a7dd87d62587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:30.973012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ac147-b71e-4d05-b884-a7dd87d62587 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:30.973253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ac147-b71e-4d05-b884-a7dd87d62587 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:30.973519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ac147-b71e-4d05-b884-a7dd87d62587 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:30.983725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ac147-b71e-4d05-b884-a7dd87d62587 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:30.985853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ac147-b71e-4d05-b884-a7dd87d62587 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-da6ac147-b71e-4d05-b884-a7dd87d62587 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:30.999956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:31.001077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:31.002290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 218/431] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:31.002970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071a5b1-83ae-40da-9b35-6b62ba38b2df None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.002970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071a5b1-83ae-40da-9b35-6b62ba38b2df None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.003129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071a5b1-83ae-40da-9b35-6b62ba38b2df None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.004824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 219/432] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:31.009029 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071a5b1-83ae-40da-9b35-6b62ba38b2df None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.010786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9e3ed-d1bd-42d5-99a5-f57e4dcc13ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.011420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9e3ed-d1bd-42d5-99a5-f57e4dcc13ff None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.011695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9e3ed-d1bd-42d5-99a5-f57e4dcc13ff None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.011963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9e3ed-d1bd-42d5-99a5-f57e4dcc13ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.022933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9e3ed-d1bd-42d5-99a5-f57e4dcc13ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.024447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9e3ed-d1bd-42d5-99a5-f57e4dcc13ff None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b1a9e3ed-d1bd-42d5-99a5-f57e4dcc13ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:31.039450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:31.041986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:31.044015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:31.044015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:31.044015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:31.044015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 214/433] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:31.044015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7831c2-77e7-4ea1-bfd1-b6f2a43e188d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.044015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7831c2-77e7-4ea1-bfd1-b6f2a43e188d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.044015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7831c2-77e7-4ea1-bfd1-b6f2a43e188d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.065928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4d7831c2-77e7-4ea1-bfd1-b6f2a43e188d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:31.067283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4d7831c2-77e7-4ea1-bfd1-b6f2a43e188d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:31.110630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.127301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071a5b1-83ae-40da-9b35-6b62ba38b2df None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.135075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.146191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 215/434] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:31.147122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7831c2-77e7-4ea1-bfd1-b6f2a43e188d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.170025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.170630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.170899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.171159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.171870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:31.172165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.173875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.198601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 220/435] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:31.198601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.301069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7831c2-77e7-4ea1-bfd1-b6f2a43e188d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.324924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5ea3c-c279-4ad8-a28d-636c1f039fc5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.347791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.348423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.348700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.348969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.349706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:31.350005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.351363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.364657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.365283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.365389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 216/436] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:31.365558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.365820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.366292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.378724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:31.384581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:31.385452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:31.386231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 221/437] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:31.386885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.402439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.426675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.477918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc5e9f-188c-4df6-aa40-b24c76fe0f8d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.501043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe87b5-969e-4f19-8eac-2bc2065a8f8f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.510210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.510825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.511095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.511358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.522062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.523535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.534403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bea17-b912-4dba-a441-6bb579bb06e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.535013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bea17-b912-4dba-a441-6bb579bb06e4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.535284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bea17-b912-4dba-a441-6bb579bb06e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.535550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bea17-b912-4dba-a441-6bb579bb06e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:31.537800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:31.538839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:31.539735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:31.539735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:31.539735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:31.539735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 217/438] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:31.539735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.545780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bea17-b912-4dba-a441-6bb579bb06e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.547201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bea17-b912-4dba-a441-6bb579bb06e4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d52bea17-b912-4dba-a441-6bb579bb06e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:31.558946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:31.559812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 222/439] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:31.561232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabff44-862d-4d1d-95a9-4e20f1f440a5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.561971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabff44-862d-4d1d-95a9-4e20f1f440a5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.561971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabff44-862d-4d1d-95a9-4e20f1f440a5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.562828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 223/440] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:31.563502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabff44-862d-4d1d-95a9-4e20f1f440a5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.623361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.641943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1824872-8403-43d7-9f13-96a9f4374625 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.650815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.667226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabff44-862d-4d1d-95a9-4e20f1f440a5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.674426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.675046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.675322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.675595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.676271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:31.676570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.677920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.685862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 218/441] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:31.686533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.704105 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.704754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.705032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.705293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.716337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:31.721405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:31.722276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:31.723046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 224/442] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:31.723708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.783581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf55cb-8400-4481-909c-a1e05cd09ba2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.815486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111f80c-65f7-42f8-8220-a6204f863cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.816017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111f80c-65f7-42f8-8220-a6204f863cbe None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.816323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111f80c-65f7-42f8-8220-a6204f863cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.816595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111f80c-65f7-42f8-8220-a6204f863cbe None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.817220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111f80c-65f7-42f8-8220-a6204f863cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:31.817518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111f80c-65f7-42f8-8220-a6204f863cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.818866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111f80c-65f7-42f8-8220-a6204f863cbe None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.822623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700b6d0-7d3a-4a04-bdd8-8103833441be None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:31.830558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 219/443] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:31.831692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbad33-ad5d-405c-90ab-a5ec66aacc6a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.831969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbad33-ad5d-405c-90ab-a5ec66aacc6a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.832252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbad33-ad5d-405c-90ab-a5ec66aacc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.839656 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.850353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1fcbad33-ad5d-405c-90ab-a5ec66aacc6a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:31.851518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1fcbad33-ad5d-405c-90ab-a5ec66aacc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:31.856088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:31.856705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:31.856974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:31.857325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:31.857941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:31.858234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:31.859647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:31.876862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 225/444] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:31.877702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.880420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:31.918554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 220/445] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:31.919569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbad33-ad5d-405c-90ab-a5ec66aacc6a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:31.988416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f6320-946b-4a04-94c9-402452df7f9e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.022785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.023402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.023672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.023936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.028641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbad33-ad5d-405c-90ab-a5ec66aacc6a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.033606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.035027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:32.052351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:32.053454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:32.054483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:32.054483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:32.054483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:32.060166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 226/446] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:32.061061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.069078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.069624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.069898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.070164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.071827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:32.072120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.073860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.075479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.083297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 221/447] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:32.084141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.124886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.170284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15568656-a4cf-46aa-b757-a104f2c2f794 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.197917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc852a4-860c-44f4-9cdb-be15b897ca20 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.205153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.205777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.206052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.206320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.212845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.214261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:32.225088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:32.226017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:32.226852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 227/448] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:32.227894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.235145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.235741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.236014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.236339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.296677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.303342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.304816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.327215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22043d5-dc21-4301-a944-7f897532997a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.340009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.360466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.361086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.361359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.361625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.362245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:32.362540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.363884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.376442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 228/449] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:32.377220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.400814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 222/450] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:32.401631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.477708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586dd33-fe5b-4d14-9a1e-c72471b3f4d6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.492278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.498737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a407-0ec4-4ccc-ae39-184f8500a852 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.511519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.512139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.512434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.512697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.522589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.524017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.531001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.531610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.531880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.532142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:32.537960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:32.538873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:32.540567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:32.540567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:32.540567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:32.540567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 229/451] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:32.540567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.546589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:32.551349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:32.552209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:32.553065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 223/452] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:32.553710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.568463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.639357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ff753-9dfd-47e2-8f01-c7b2a83e9189 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.655541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfafedd-454b-4ed5-80e4-4b464a275409 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.675626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.676259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.676531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.676801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.677476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:32.677773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.679185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.687871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.688508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.688775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.689064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.690937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 230/453] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:32.692187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.696195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.697653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:32.708890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:32.709804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 224/454] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:32.710664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.731874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.773119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.787844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba98e3-4483-4dcb-a907-ceccf8cd714f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.812465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9421c6c-4988-4589-ae19-a3b822f52984 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.820572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.821228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.821496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.821762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.831191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.832640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.844892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.845479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.845778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.846044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.846670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:32.846963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:32.847428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:32.848450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:32.849292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:32.849292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:32.849292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:32.849455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.849530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 231/455] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:32.849767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.860457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 225/456] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:32.861496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.935418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.946345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13b02e-b7d8-47d1-8436-f320d3c82537 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.957914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21c127-1d63-422a-8929-54c816846ff0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:32.979902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.980526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.980799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.981069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:32.981712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:32.982005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:32.983355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:32.988176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:32.991469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 232/457] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:32.992023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:32.992328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:32.992580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:32.992881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:32.993154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/edb1a2c4a51f46c1be39baf0927eff4a/roles/eb326c809f1543ce8be84f07a0a1cdab` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.010532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=edb1a2c4a51f46c1be39baf0927eff4a, role_id=eb326c809f1543ce8be84f07a0a1cdab)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.011949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.023851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 226/458] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:32 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/edb1a2c4a51f46c1be39baf0927eff4a/roles/eb326c809f1543ce8be84f07a0a1cdab => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:33.025212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.107950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535bc1e-47e4-45d5-9b07-e2fabb68de03 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.147624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.148245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.148556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.148824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.150179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:33.150465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.151224 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77127c-8e52-472c-aed9-8845d045fdd3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.151923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.176547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 233/459] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Oct 27 13:11:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:33.176547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.184974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.197732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.199191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.199191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.199191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.199527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:33.199823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.201916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.216335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 227/460] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:33.217319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.242202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.292496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958f47-f8b7-493b-97cf-46b49ad94efe None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.317171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd231ae-0c5b-413a-a2d3-418b380ad79c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.328090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.328821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.329157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.329531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.337274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.338776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:33.350787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:33.351847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 27 13:11:33.353123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:33.358376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 234/461] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:33.358376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.359924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.360676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.361047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.361408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.373043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.374595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:33.390082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:33.391159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:33.392571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:33.392571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:33.392571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:33.392571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 228/462] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:33.392783 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.431642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.471840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b68157-29e8-45ed-a8eb-f27feb432621 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.489858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.490123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70223dbc-bce8-4091-b879-c3f7836e0c38 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.512343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.513174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.513447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.513713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.527533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.528151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.528474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.528755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.535360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.536823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:33.548462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:33.549270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:33.550097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 229/463] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:33.551317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.589678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.591292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.653708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.657754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35402e3b-f978-454e-95de-79f600150bfc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.695310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.696616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.696962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.697233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.697946 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:33.698120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 235/464] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:33.698949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.702681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.704118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.718343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.722941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 230/465] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:33.728473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.806890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e7dac-3e7f-49c2-a48b-5b684b4d9730 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.836564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e4407-f4bc-49ff-8a37-67daae5af594 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.840642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.841296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.841574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.841839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.842466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:33.842761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.844113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.855680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 236/466] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:33.857101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.873396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:33.874005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:33.874284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:33.874552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:33.875238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:33.875540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:33.876973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:33.893314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 231/467] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:33.893314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:33.916171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.949827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:33.976622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761ed44-f5af-4fa9-a7f9-dd0bb9a792a2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:33.998781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc720794-0fdf-409c-8b6a-8e3d99624253 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.018219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.019353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.019632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.020173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.031936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.032561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.032834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.033058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.033167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.034880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:34.051183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:34.052051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:34.052931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:34.052931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:34.052931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:34.052931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 237/468] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:34.053147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.112362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.113785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.131133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:34.160287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36ca17-c1e5-4d70-b473-07328c6472a4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.187805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:34.194110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.194739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.195018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.195290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.203038 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.204491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:34.216379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:34.217428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 238/469] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:34.218304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.219735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 232/470] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:34.220722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.326976 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0c723-7810-4566-ad62-c18f0e8878f1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.332569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9fa6-9557-47bf-9d90-205d157d0d07 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.345190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:34.364639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.365459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.365730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.365993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.366613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:34.366904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.368295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.368663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.369261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.369539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.369808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.370522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] RBAC: Adding query filter params (role.id=eb326c809f1543ce8be84f07a0a1cdab, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=d1d142bc3a9e4b3badefe713c7405329) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:34.370822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.372299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.379996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 233/471] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:34.384957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.386855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 239/472] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/role_assignments?user.id=d1d142bc3a9e4b3badefe713c7405329&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=eb326c809f1543ce8be84f07a0a1cdab => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:34.387965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.469824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:34.506566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694ecae-1aee-4546-9ac4-61547ff23fa2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.537351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa3663-b3e7-4acd-86d6-261effa9a456 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.550082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.551068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.551559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.552015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.553483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:34.554130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.555709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.577571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 234/473] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:34.577916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.585009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.585949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.586232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.587115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.606093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.607061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.626600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:34.628543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:34.629689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:34.630525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:34.630525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:34.630525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:34.634885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 240/474] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:34.635775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.718083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d55d7-fc59-4014-80b0-17b49e873d23 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.754527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64193271-bfbf-434a-9035-3ef32f9ce922 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.763845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:34.768316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.769240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.769661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.769939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.780038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.781802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.794582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.795189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.795465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.795787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:34.797543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:34.798337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:34.799164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:34.800120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 235/475] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:34.800974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:34.879833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.881356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.924918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:34.935275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791094-3462-44bb-96ab-b86f9b61ab93 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:34.971161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:34.971811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:34.972092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:34.972409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:34.973050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:34.973405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:34.974877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:34.991683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 236/476] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:34.992642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.008370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 241/477] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:35.009357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.020435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:35.116894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b29d7-df2d-4ceb-839e-9f0f2702a3f4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.135822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887e46f-ef5f-47b8-bc24-44c2a0bf6dd6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.164174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.165792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.166076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.166348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.167318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:35.167623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.169888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.173845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871caae-a23b-4684-a5fe-caa2a917cd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.174617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871caae-a23b-4684-a5fe-caa2a917cd52 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.174995 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871caae-a23b-4684-a5fe-caa2a917cd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.175361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871caae-a23b-4684-a5fe-caa2a917cd52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.182492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871caae-a23b-4684-a5fe-caa2a917cd52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.184081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871caae-a23b-4684-a5fe-caa2a917cd52 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.184826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 237/478] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:35.185767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9871caae-a23b-4684-a5fe-caa2a917cd52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:35.202049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:35.202954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:35.203778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 242/479] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:35.205314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:35.209360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6028-e82e-45f4-a020-49e0d992bca4 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.209646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6028-e82e-45f4-a020-49e0d992bca4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.209919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6028-e82e-45f4-a020-49e0d992bca4 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.215931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 243/480] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:35.215931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6028-e82e-45f4-a020-49e0d992bca4 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.282067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:35.296779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f281ec3-37ce-4811-aa58-b20843cf9d8a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.321697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6028-e82e-45f4-a020-49e0d992bca4 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.329772 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.330418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.330690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.330951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.342427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.343881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:35.358423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:35.359307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:35.364929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:35.364929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:35.364929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:35.364929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 238/481] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:35.364929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.368036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b74906-e626-4b51-adc3-86c7829b96a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.368933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74906-e626-4b51-adc3-86c7829b96a1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.369474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74906-e626-4b51-adc3-86c7829b96a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.369745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74906-e626-4b51-adc3-86c7829b96a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.370460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b74906-e626-4b51-adc3-86c7829b96a1 None admin] RBAC: Adding query filter params (role.id=eb326c809f1543ce8be84f07a0a1cdab, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=80a57dd6e99c4cac9e6bcd1413afd5c0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:35.370762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b74906-e626-4b51-adc3-86c7829b96a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.372311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b74906-e626-4b51-adc3-86c7829b96a1 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.384332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 244/482] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/role_assignments?user.id=80a57dd6e99c4cac9e6bcd1413afd5c0&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=eb326c809f1543ce8be84f07a0a1cdab => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:35.384332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cd43f-eee0-4291-b117-93b1b60345f1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.384332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cd43f-eee0-4291-b117-93b1b60345f1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.384332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cd43f-eee0-4291-b117-93b1b60345f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.407877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-105cd43f-eee0-4291-b117-93b1b60345f1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:35.409059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-105cd43f-eee0-4291-b117-93b1b60345f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:35.444449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:35.495975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d58a-4f76-4f78-bf85-d18c408179ac None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.501852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 245/483] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:35.502732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cd43f-eee0-4291-b117-93b1b60345f1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.576175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.576175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.576175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.576175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.576558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:35.576558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.576558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.590743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 239/484] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:35.599120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.604340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:35.649813 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cd43f-eee0-4291-b117-93b1b60345f1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.694619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.695479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.695877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.696577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.697300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:35.697675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.699536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.715180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 246/485] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:35.717556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b90bd-c455-4f35-90bd-c36f22049277 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.721859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.747444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:35.758267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.759136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.759591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.760064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.760997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] RBAC: Adding query filter params (role.id=eb326c809f1543ce8be84f07a0a1cdab, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=b7cba7b6049e41488f5b2f7c61a0500a) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:35.761511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.763124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.770617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 240/486] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/role_assignments?user.id=b7cba7b6049e41488f5b2f7c61a0500a&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=eb326c809f1543ce8be84f07a0a1cdab => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:35.815266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:35.863697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726332fa-bf86-4495-be8d-a42b55a0f7ca None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:35.910896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:35.952750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:35.953954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:35.955983 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:35.957400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:35.970347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:35.972220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:35.985495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:35.986398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:35.987253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:35.994071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 247/487] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:35.996684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:36.010522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9991e-98a7-4bdc-8a17-562816543e9d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:36.062280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:36.062856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.063131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.063394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.074824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:36.079558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:36.080477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:36.081668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 241/488] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:36.086837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:36.110880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf34a6-b5ec-4291-a714-51d9aad8cdf4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:36.145327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:36.148561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:36.149496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.150040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.150511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.151399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:36.151902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:36.153563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:36.172136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 248/489] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:36.173707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:36.207941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bb6ca-1138-4c4f-b4b6-8b343968b570 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:36.224636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:36.247159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:36.247734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.248027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.248331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.248989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:36.249302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:36.250790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:36.260041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 242/490] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:36.265603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:36.293513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1924d3e-2a74-44c9-8177-c1f9e39f9c73 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:36.331643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:36.332632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.333140 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.333682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.334625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] RBAC: Adding query filter params (role.id=eb326c809f1543ce8be84f07a0a1cdab, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=2d4ce3e02e5945ffbbde389398b146c4) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:36.335131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:36.337334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:36.348356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 249/491] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:36 2022] GET /identity/v3/role_assignments?user.id=2d4ce3e02e5945ffbbde389398b146c4&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=eb326c809f1543ce8be84f07a0a1cdab => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:36.444318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e0e93-7aaf-44f0-a40e-a59e1f08be60 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:36.466565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:36.493660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:36.494302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.494576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.494842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.507153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:36.508678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:36.524993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:36.525964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:36.527843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:36.527843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:36.527843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:36.527843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 243/492] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:36.533402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:36.646650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c774f-0034-411d-8182-935c6a2e83ae None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:36.661354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:36.684424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f5044-6087-43a1-a06b-f5101f830c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:36.685372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f5044-6087-43a1-a06b-f5101f830c4e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.685941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f5044-6087-43a1-a06b-f5101f830c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.686412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f5044-6087-43a1-a06b-f5101f830c4e None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.687259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f5044-6087-43a1-a06b-f5101f830c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:36.687758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f5044-6087-43a1-a06b-f5101f830c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:36.689345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f5044-6087-43a1-a06b-f5101f830c4e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:36.702809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 250/493] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:36.709734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:36.758686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f17783-b628-4497-bb87-2914a9db0945 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.759277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f17783-b628-4497-bb87-2914a9db0945 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.759751 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f17783-b628-4497-bb87-2914a9db0945 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.761795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 251/494] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:36.767963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ee4e6-f418-4ef6-b445-dceaa0666b47 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.768320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ee4e6-f418-4ef6-b445-dceaa0666b47 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.768520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ee4e6-f418-4ef6-b445-dceaa0666b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.789988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-fc2ee4e6-f418-4ef6-b445-dceaa0666b47 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:36.791339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fc2ee4e6-f418-4ef6-b445-dceaa0666b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:36.832411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9dba6-7b7d-4683-9fc5-4a6f34de27ee None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:36.873322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 252/495] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:36.876710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:36.877492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:36.877921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:36.878315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:36.890754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:36.892711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:36.904378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:36.905278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:36.906231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:36.908866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 244/496] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:36.915387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ee4e6-f418-4ef6-b445-dceaa0666b47 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:37.038341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ee4e6-f418-4ef6-b445-dceaa0666b47 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:37.044564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:37.091175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:37.092107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:37.092583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:37.093027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:37.093921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:37.094400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:37.096013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:37.114576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 253/497] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Oct 27 13:11:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:37.120159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:37.159735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:37.262271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ce86-a674-42c1-ab13-ebc8659285ee None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:37.292906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:37.312220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:37.312840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:37.313112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:37.313382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:37.433179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:37.434635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:37.463620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:37.470392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fef11-45ee-4faa-83db-80070858eb02 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:37.547847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 245/498] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 27 13:11:37.552837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:37.556970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:37.557605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:37.557875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:37.558134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:37.573834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:37.579309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:37.580275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:37.581120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 254/499] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:37.587059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:37.612263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:37.709383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38733-fd28-4142-b8bc-16d71ea6f66e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:37.748117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:37.748758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:37.749033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:37.749299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:37.750240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:37.750537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:37.752009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:37.761863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886c3b9-6b64-4de9-ae29-65dd07e90ac8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:37.762227 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 255/500] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:37.768494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:37.821884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:37.853484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:37.857555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:37.858339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:37.858612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:37.858877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:37.881599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033198-577b-4641-9a3c-c8f33a7e57a3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:37.885263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:37.886706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:37.916754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:37.917374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:37.917646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:37.917913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:37.921741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:37.923101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:37.923962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:37.923962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:37.923962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:37.923962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 246/501] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:37.928999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:37.931008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:37.933156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:37.946991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:37.949201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:37.951261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:37.951261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:37.951261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:37.954647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 256/502] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:37.954647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:38.062425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4b055-5c19-4798-9609-972a6f465867 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:38.113513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:38.127336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:38.128755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.129310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.129816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.130654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:38.131149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:38.132735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:38.157956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 257/503] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:38.163655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:38.171834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fcc04-39c6-4d4b-9c40-577297912b2e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:38.226485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:38.227664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.228922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.229921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.232798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:38.233356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:38.234910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:38.259356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 247/504] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:38.263485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:38.313730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:38.332878 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25439-bb7f-420a-b815-30d564c5a6d2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:38.377207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1462ae-c45c-42c6-b301-2058354f8da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:38.378403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1462ae-c45c-42c6-b301-2058354f8da7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.378825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1462ae-c45c-42c6-b301-2058354f8da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.379532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1462ae-c45c-42c6-b301-2058354f8da7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.390146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1462ae-c45c-42c6-b301-2058354f8da7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:38.391623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1462ae-c45c-42c6-b301-2058354f8da7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:38.393955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b6a81-ec8b-4081-be03-00bd6c1ea784 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3e1462ae-c45c-42c6-b301-2058354f8da7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:38.409452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:38.410438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:38.411466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:38.414332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 258/505] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Oct 27 13:11:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:38.416013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcf41c-60dc-44bd-9025-eeaec3b2f197 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.416748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcf41c-60dc-44bd-9025-eeaec3b2f197 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.417162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcf41c-60dc-44bd-9025-eeaec3b2f197 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.423753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 259/506] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:38.425156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcf41c-60dc-44bd-9025-eeaec3b2f197 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:38.439394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f2e0e-26c5-4e2d-a52c-985e406bea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:38.440150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f2e0e-26c5-4e2d-a52c-985e406bea9d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.440565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f2e0e-26c5-4e2d-a52c-985e406bea9d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.441133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f2e0e-26c5-4e2d-a52c-985e406bea9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.449041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f2e0e-26c5-4e2d-a52c-985e406bea9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:38.450681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f2e0e-26c5-4e2d-a52c-985e406bea9d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-e35f2e0e-26c5-4e2d-a52c-985e406bea9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:38.463712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:38.464764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:38.465820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:38.467269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 248/507] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:38.472247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fb33-5012-4866-9f5f-24926fd85b62 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.472247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fb33-5012-4866-9f5f-24926fd85b62 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.472247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fb33-5012-4866-9f5f-24926fd85b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.489262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:38.490837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5138fb33-5012-4866-9f5f-24926fd85b62 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:38.492312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5138fb33-5012-4866-9f5f-24926fd85b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:38.552471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcf41c-60dc-44bd-9025-eeaec3b2f197 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:38.572259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 249/508] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:38.572259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fb33-5012-4866-9f5f-24926fd85b62 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:38.605386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a894500-4c29-4dca-a412-45c35a2c1122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:38.606566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a894500-4c29-4dca-a412-45c35a2c1122 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.606973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a894500-4c29-4dca-a412-45c35a2c1122 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.607426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a894500-4c29-4dca-a412-45c35a2c1122 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.608698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a894500-4c29-4dca-a412-45c35a2c1122 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:38.609251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a894500-4c29-4dca-a412-45c35a2c1122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:38.611965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a894500-4c29-4dca-a412-45c35a2c1122 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:38.629121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:38.645208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 260/509] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Oct 27 13:11:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:38.647637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b055f79-bd01-4812-b9a0-89b3b8bf5db6 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.648339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b055f79-bd01-4812-b9a0-89b3b8bf5db6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.649059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b055f79-bd01-4812-b9a0-89b3b8bf5db6 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.651760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 261/510] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:38.653460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b055f79-bd01-4812-b9a0-89b3b8bf5db6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:38.764396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fb33-5012-4866-9f5f-24926fd85b62 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:38.786039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b055f79-bd01-4812-b9a0-89b3b8bf5db6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:38.788577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:38.825154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424f736-b6d8-4137-aee3-4826cd2a8b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:38.825898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424f736-b6d8-4137-aee3-4826cd2a8b92 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.826170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424f736-b6d8-4137-aee3-4826cd2a8b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.826548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424f736-b6d8-4137-aee3-4826cd2a8b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.830697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:38.831801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.832428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.833004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.834492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:38.835122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:38.837739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:38.837910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424f736-b6d8-4137-aee3-4826cd2a8b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:38.839287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424f736-b6d8-4137-aee3-4826cd2a8b92 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-0424f736-b6d8-4137-aee3-4826cd2a8b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:38.854586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:38.856563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:38.858358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:38.858358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:38.858358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:38.858358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 262/511] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:38.858358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f7f4d-bc81-48ef-83f0-5b3c3da9f8e8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:38.858358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f7f4d-bc81-48ef-83f0-5b3c3da9f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:38.858358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f7f4d-bc81-48ef-83f0-5b3c3da9f8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:38.864665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 250/512] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:38.868932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:38.882532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-014f7f4d-bc81-48ef-83f0-5b3c3da9f8e8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:38.884767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-014f7f4d-bc81-48ef-83f0-5b3c3da9f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:38.961457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 263/513] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:38.962301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f7f4d-bc81-48ef-83f0-5b3c3da9f8e8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.021883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.040345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dac385-dbef-4e2e-b9ec-00c63d479096 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.080417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.081252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.081624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.081981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.082478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.083724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:39.084119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.086055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.098698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f7f4d-bc81-48ef-83f0-5b3c3da9f8e8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.099697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 251/514] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:39.100515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.135990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0178e3-5554-42d8-a105-d60679b36e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.136576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0178e3-5554-42d8-a105-d60679b36e85 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.136576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0178e3-5554-42d8-a105-d60679b36e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.136814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0178e3-5554-42d8-a105-d60679b36e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.137557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0178e3-5554-42d8-a105-d60679b36e85 None admin] RBAC: Adding query filter params (role.id=eb326c809f1543ce8be84f07a0a1cdab, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=edb1a2c4a51f46c1be39baf0927eff4a) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:39.137859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0178e3-5554-42d8-a105-d60679b36e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.139258 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0178e3-5554-42d8-a105-d60679b36e85 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.145115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 264/515] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:38 2022] GET /identity/v3/role_assignments?user.id=edb1a2c4a51f46c1be39baf0927eff4a&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=eb326c809f1543ce8be84f07a0a1cdab => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:39.146233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238940b5-2b48-4836-b22b-d7cf9be1d6e0 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.146508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238940b5-2b48-4836-b22b-d7cf9be1d6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.146774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238940b5-2b48-4836-b22b-d7cf9be1d6e0 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.152042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 265/516] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:39.152042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238940b5-2b48-4836-b22b-d7cf9be1d6e0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.231164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6ec24-2e9a-438e-9b2c-b749dc4f5033 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.272858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238940b5-2b48-4836-b22b-d7cf9be1d6e0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.274414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa186ee-aad6-402c-a70f-94111c5fa625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.275052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa186ee-aad6-402c-a70f-94111c5fa625 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.275324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa186ee-aad6-402c-a70f-94111c5fa625 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.275596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa186ee-aad6-402c-a70f-94111c5fa625 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.276185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa186ee-aad6-402c-a70f-94111c5fa625 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.277580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa186ee-aad6-402c-a70f-94111c5fa625 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-4fa186ee-aad6-402c-a70f-94111c5fa625 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:39.290740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:39.291793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:39.292712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 252/517] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:39.293523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21376-b5e3-45a8-b206-f390f6a7f7ca None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.294372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21376-b5e3-45a8-b206-f390f6a7f7ca None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.294372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21376-b5e3-45a8-b206-f390f6a7f7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.313163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a8d21376-b5e3-45a8-b206-f390f6a7f7ca None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:39.314369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a8d21376-b5e3-45a8-b206-f390f6a7f7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:39.316609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.317249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.317520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.317785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.318616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.325387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.326845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:39.340339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:39.341436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 266/518] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:39.342301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.355277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.408341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 253/519] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:39.408341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d21376-b5e3-45a8-b206-f390f6a7f7ca None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.457596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73195b0-9d78-4e65-92a8-e552bbbeab01 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.500293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.500928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.501602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.501971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] PATH_INFO: `/v3/services/network` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.502831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.504472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:39.524145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:39.525281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:39.526111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:39.526860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d21376-b5e3-45a8-b206-f390f6a7f7ca None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.534093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 267/520] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:39.534093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.572757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.573410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.573683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.573947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.574564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:39.574857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.576209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.585952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 254/521] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:39.587475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.591516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.634897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.667251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1af60-bb52-4bf5-917b-0c2592b59d53 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.707956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.708570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2347f-8ea8-4468-aa65-8a2a4d6b8995 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.708682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.708854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.709118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.709793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:39.710259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.712814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.731015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 268/522] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:39.731015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.748276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.748886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.749157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.749723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.750027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] RBAC: Adding query filter params (name=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:39.750321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.751663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.756170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 255/523] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:39.761133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.828974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.837850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c41d8-5cd9-41a9-aa17-90fedab014d8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.863833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef832db-673d-478f-8de9-023ec6e31230 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:39.875654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.876302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.876609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.876845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.877475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:39.877757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.879768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.881106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:39.892278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 269/524] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:39.892278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:39.912558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:39.913197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:39.913472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:39.913737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:39.929486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:39.931823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:39.952107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:39.953214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:39.954088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:39.954088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:39.954088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:39.956411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 256/525] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:39.957671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:40.019147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e4deb-82dd-4cd2-b388-a46ac1455c23 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:40.056807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:40.057474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:40.057749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:40.058132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/d1d142bc3a9e4b3badefe713c7405329/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:40.060021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=d1d142bc3a9e4b3badefe713c7405329, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:40.061585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:40.088347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4596da-05bf-47ec-8b1c-30726ea247ed None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:40.090291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 270/526] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:39 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/d1d142bc3a9e4b3badefe713c7405329/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:40.090291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:40.121561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:40.145082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:40.146913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:40.147370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:40.147789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:40.148701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] RBAC: Adding query filter params (type=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:40.150353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:40.152892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:40.168883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 257/527] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:11:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:40.177827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:40.233911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387c7a1-66fb-4c20-8332-f0447a36149f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:40.288463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4a6cb-c220-4be9-8148-6b0e1b79a251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:40.290109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a6cb-c220-4be9-8148-6b0e1b79a251 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:40.290574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a6cb-c220-4be9-8148-6b0e1b79a251 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:40.290843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a6cb-c220-4be9-8148-6b0e1b79a251 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:40.291485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a6cb-c220-4be9-8148-6b0e1b79a251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:40.291776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a6cb-c220-4be9-8148-6b0e1b79a251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:40.293196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a6cb-c220-4be9-8148-6b0e1b79a251 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:40.306971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 271/528] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:40.312697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:40.445228 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fbe85-7530-43ad-ac39-f5822b363689 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:40.497228 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f18a8-a199-4174-b208-e9a4fa562328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:40.497760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f18a8-a199-4174-b208-e9a4fa562328 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:40.498044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f18a8-a199-4174-b208-e9a4fa562328 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:40.498317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f18a8-a199-4174-b208-e9a4fa562328 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:40.498977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f18a8-a199-4174-b208-e9a4fa562328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:40.500430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f18a8-a199-4174-b208-e9a4fa562328 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:40.540403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 258/529] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:11:40 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:41.578334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d164508-39f8-4d25-8f8d-9df10865b3b2 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:41.578892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d164508-39f8-4d25-8f8d-9df10865b3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:41.578892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d164508-39f8-4d25-8f8d-9df10865b3b2 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:41.580480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 272/530] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:41.585893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207999a4-447d-4b45-87c4-ca0ea2e406fb None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:41.586920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207999a4-447d-4b45-87c4-ca0ea2e406fb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:41.586999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207999a4-447d-4b45-87c4-ca0ea2e406fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:41.623987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-207999a4-447d-4b45-87c4-ca0ea2e406fb None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:41.625198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-207999a4-447d-4b45-87c4-ca0ea2e406fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:41.670678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:41.707472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 259/531] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:42.170973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d164508-39f8-4d25-8f8d-9df10865b3b2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:42.309879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d164508-39f8-4d25-8f8d-9df10865b3b2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:42.330613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:42.350478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:42.351109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.351386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.351653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.369092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:42.373854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:42.374744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:42.375594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 273/532] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:42.381178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207999a4-447d-4b45-87c4-ca0ea2e406fb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:42.514337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207999a4-447d-4b45-87c4-ca0ea2e406fb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:42.556478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9cf78-9378-4dca-95ab-dd2342310340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:42.557081 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9cf78-9378-4dca-95ab-dd2342310340 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.557352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9cf78-9378-4dca-95ab-dd2342310340 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.557621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9cf78-9378-4dca-95ab-dd2342310340 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.558274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9cf78-9378-4dca-95ab-dd2342310340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:42.558569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9cf78-9378-4dca-95ab-dd2342310340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:42.559928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9cf78-9378-4dca-95ab-dd2342310340 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:42.572631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 260/533] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:42.577966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:42.665717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fce1ee-cb8a-4873-b452-2218c3365323 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.666251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fce1ee-cb8a-4873-b452-2218c3365323 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.666251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fce1ee-cb8a-4873-b452-2218c3365323 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.667821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 261/534] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:42.673376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a72f95-26ba-408f-90aa-0511a33bfeae None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.673711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a72f95-26ba-408f-90aa-0511a33bfeae None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.673919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a72f95-26ba-408f-90aa-0511a33bfeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.714213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-32a72f95-26ba-408f-90aa-0511a33bfeae None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:42.717506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-32a72f95-26ba-408f-90aa-0511a33bfeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:42.765872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7ed22-3a16-4c42-8001-35e9a75f41f1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:42.767271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:42.800787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46385f6c-df1b-41bf-a1b5-2d40b3298efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:42.801400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46385f6c-df1b-41bf-a1b5-2d40b3298efc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.801670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46385f6c-df1b-41bf-a1b5-2d40b3298efc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.801936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46385f6c-df1b-41bf-a1b5-2d40b3298efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.812353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46385f6c-df1b-41bf-a1b5-2d40b3298efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:42.813769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46385f6c-df1b-41bf-a1b5-2d40b3298efc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:42.825594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 262/535] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 27 13:11:42.826633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df2417-aad9-4a11-99cb-ff3dbe9b6448 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.826911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df2417-aad9-4a11-99cb-ff3dbe9b6448 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.827176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df2417-aad9-4a11-99cb-ff3dbe9b6448 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-46385f6c-df1b-41bf-a1b5-2d40b3298efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:42.828074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:42.829092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:42.834607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:42.834607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:42.834607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:42.834607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 274/536] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:42.834607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4778833-20fd-4e05-abdf-fa0e1207162a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.834607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4778833-20fd-4e05-abdf-fa0e1207162a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.834607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4778833-20fd-4e05-abdf-fa0e1207162a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.835004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 263/537] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:42.835759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db795743-5d1f-4a01-a78d-1a84fc1c4d73 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.836682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db795743-5d1f-4a01-a78d-1a84fc1c4d73 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.836793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 275/538] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:42.836878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db795743-5d1f-4a01-a78d-1a84fc1c4d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.837471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ac1f-feeb-4a1b-94f1-fba890ccc7fe None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:42.837750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ac1f-feeb-4a1b-94f1-fba890ccc7fe None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:42.838015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ac1f-feeb-4a1b-94f1-fba890ccc7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:42.858554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-db795743-5d1f-4a01-a78d-1a84fc1c4d73 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:42.859704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-db795743-5d1f-4a01-a78d-1a84fc1c4d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:42.859914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0466ac1f-feeb-4a1b-94f1-fba890ccc7fe None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:42.861108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0466ac1f-feeb-4a1b-94f1-fba890ccc7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:42.892330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:42.932645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 264/539] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:42.941384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db795743-5d1f-4a01-a78d-1a84fc1c4d73 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:42.952323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 276/540] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 27 13:11:43.077320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:43.086097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db795743-5d1f-4a01-a78d-1a84fc1c4d73 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:43.132989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:43.133624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.133896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.134161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.134796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:43.135088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:43.136456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:43.139257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ac1f-feeb-4a1b-94f1-fba890ccc7fe None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.156265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 265/541] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:43.165718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.274943 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ac1f-feeb-4a1b-94f1-fba890ccc7fe None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:43.323564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:43.324179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.324494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.324762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.325460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:43.327707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:43.333033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31a0c6-9faa-48d9-b8e8-2e6ec0faf180 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:43.351556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 277/542] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:11:43 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:11:43.361475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.399417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:43.399926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:43.400476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.400757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.401025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.412308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:43.414047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:43.428602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:43.429609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 266/543] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:43.431006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.444188 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:43.519375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4c15-19d7-410d-9b95-72b7150cea02 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:43.548414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c1105-8a9e-464a-8e63-8b7de12de5dd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:43.560976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:43.561697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.562342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.562342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.577859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:43.582523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:43.583605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:43.584787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 278/544] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:43.585685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.593537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef4a0e-4270-4ae6-a66b-c752ef19ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:43.594370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef4a0e-4270-4ae6-a66b-c752ef19ac96 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.594797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef4a0e-4270-4ae6-a66b-c752ef19ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.595213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef4a0e-4270-4ae6-a66b-c752ef19ac96 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.596042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef4a0e-4270-4ae6-a66b-c752ef19ac96 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:43.597441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef4a0e-4270-4ae6-a66b-c752ef19ac96 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:43.629771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 267/545] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:11:43 2022] POST /identity/v3/services => generated 240 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:43.629771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179f986-2fc5-43dc-8a7f-331c3cc3a108 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.629771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179f986-2fc5-43dc-8a7f-331c3cc3a108 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.629771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179f986-2fc5-43dc-8a7f-331c3cc3a108 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.629771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 268/546] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:43.629771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179f986-2fc5-43dc-8a7f-331c3cc3a108 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.679017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:43.703043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37deed-a079-43a5-80bc-5b21582e9dea None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:43.722668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:43.755070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c68e29-e056-4b8a-a718-c4d0f87fa712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:43.755795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c68e29-e056-4b8a-a718-c4d0f87fa712 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.756078 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c68e29-e056-4b8a-a718-c4d0f87fa712 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.756377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c68e29-e056-4b8a-a718-c4d0f87fa712 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.757162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c68e29-e056-4b8a-a718-c4d0f87fa712 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:43.757465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c68e29-e056-4b8a-a718-c4d0f87fa712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:43.759509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c68e29-e056-4b8a-a718-c4d0f87fa712 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:43.768839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179f986-2fc5-43dc-8a7f-331c3cc3a108 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:43.780018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 279/547] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Oct 27 13:11:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:43.786101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243055-92dc-42d9-af59-b138ea135148 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.786426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243055-92dc-42d9-af59-b138ea135148 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.786656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243055-92dc-42d9-af59-b138ea135148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.805852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e8243055-92dc-42d9-af59-b138ea135148 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:43.807083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e8243055-92dc-42d9-af59-b138ea135148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:43.812979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:43.813640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:43.813917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:43.814183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:43.814854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:43.815151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:43.817279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:43.836800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 269/548] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:43.836800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.898933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 280/549] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:43.900266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8243055-92dc-42d9-af59-b138ea135148 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:43.959318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae4117-372e-4f67-9c44-85cf68da0947 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.000434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.017048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.026455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.027538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.027960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.028676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.047948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.049505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.064202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8243055-92dc-42d9-af59-b138ea135148 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:44.079196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:44.080170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:44.081088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:44.081088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:44.081088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:44.082343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 270/550] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:44.085735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.127308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.128241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.128657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.129063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.146160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.147622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:44.175260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:44.176272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:44.177158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:44.177158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:44.177158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:44.183769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 281/551] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:44.191513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.227977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a5cb-36ee-4786-b392-483684c8285c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.270549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.276283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.276283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.276283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.276283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:44.276283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.276283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.295943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 271/552] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:44.297198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.335866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944ac91-b127-47ce-9db6-a143ebdbcc63 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.353428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.361154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.373597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.374198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.374473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.374739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.375366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:44.375663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.377051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.389531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 282/553] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:44.390386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.433993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c5cd9-eac0-48d2-853a-9727ccee0aeb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.473433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.474845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.475348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.476246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.490741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:44.498658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:44.499620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:44.500479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:44.506126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 272/554] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:44.506126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.506715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344723d-cd40-43c1-9a98-17280c23409c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.542578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.543208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.543489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.543758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.550845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.552311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:44.564547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:44.565436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:44.566331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:44.570396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 283/555] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:44.571295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.607882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.637601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5602bbb-ca97-489e-bd25-c90c03bfa558 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.639474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.676795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519ad91-672e-49e4-8e0f-fcb68632c5f0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.679196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.680084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.680640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.681142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.689038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.690874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:44.705041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:44.705994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:11:44.706901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:44.709768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 273/556] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:44.710738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.712430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.713039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.713315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.713584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.714241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:44.714913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.716322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.725067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 284/557] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:44.730610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.825803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.829529 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e227ec-a4d3-4600-8f1f-5787a7883645 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.830273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737962d-59f0-4e1e-a93f-54575915b05d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:44.864996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.865600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.865881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.866152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.866864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:44.867165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.868716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.869964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:44.870839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:44.871377 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:44.871843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:44.872918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:44.873431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:44.875022 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:44.886962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 285/558] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:44.887908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.888524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 274/559] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:44.892278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:44.918940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:44.994776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ef24-5000-48f1-9701-a8de0e017db0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.002525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931d1e-a7a3-4fee-a503-735cd099badc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.032465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.033090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.033361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.033626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.041296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.042257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.042752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.043235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.043947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.045395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:45.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:45.061792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:45.062722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:45.062722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:45.062722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:45.066578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 286/560] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:45.067393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:45.084593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:45.118613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.120425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.145024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:45.175872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e0a72-12f7-4276-9ecf-7b9afa4c2254 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.213387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.213998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.214285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.214535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/edb1a2c4a51f46c1be39baf0927eff4a/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.216480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=edb1a2c4a51f46c1be39baf0927eff4a, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.217997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.227191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 275/561] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:11:45.228489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:45.231230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 287/562] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:45 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/edb1a2c4a51f46c1be39baf0927eff4a/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:45.232872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:45.368669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:45.419699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd06434-691d-4d0e-9bb6-d9a466fd3d23 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.429046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46484b97-5449-4450-983e-b75948f09cb6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.456573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:45.467776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.468442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.468717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.468982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.474880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.475750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.476280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.476819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.477791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:45.478292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.480034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.481905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.483357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.494587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 276/563] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:45.499543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:45.500720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:45.501613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:45.501613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:45.501613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:45.501780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:45.502778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 288/564] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:45.513239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:45.618837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db464903-24bc-4789-a27c-1d51fe263843 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.656079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29364704-88d5-4a69-bb9b-766e21e064f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.657062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29364704-88d5-4a69-bb9b-766e21e064f5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.657536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29364704-88d5-4a69-bb9b-766e21e064f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.658002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29364704-88d5-4a69-bb9b-766e21e064f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.666317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29364704-88d5-4a69-bb9b-766e21e064f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.668132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29364704-88d5-4a69-bb9b-766e21e064f5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-29364704-88d5-4a69-bb9b-766e21e064f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:45.681791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:45.683145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:11:45.684332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:45.686480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 277/565] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:11:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:45.687888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75a2fe-4111-48c2-9abc-d547884e6cbc None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.688327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75a2fe-4111-48c2-9abc-d547884e6cbc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.688731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75a2fe-4111-48c2-9abc-d547884e6cbc None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.690491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 278/566] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:45.693173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75a2fe-4111-48c2-9abc-d547884e6cbc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:45.740248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea943f08-5033-4a07-8ed8-c782afbb7c76 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.754461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:45.770125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:45.786379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb0252-1c1d-4b91-8771-0d01c2fe70dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.787217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb0252-1c1d-4b91-8771-0d01c2fe70dd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.787667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb0252-1c1d-4b91-8771-0d01c2fe70dd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.788079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb0252-1c1d-4b91-8771-0d01c2fe70dd None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.788880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb0252-1c1d-4b91-8771-0d01c2fe70dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:45.789858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb0252-1c1d-4b91-8771-0d01c2fe70dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.791373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb0252-1c1d-4b91-8771-0d01c2fe70dd None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.805609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 289/567] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:45.807110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2ee6-9897-4d31-ad42-a1900467acfb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.807534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2ee6-9897-4d31-ad42-a1900467acfb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.807948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2ee6-9897-4d31-ad42-a1900467acfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.823138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75a2fe-4111-48c2-9abc-d547884e6cbc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:45.826308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2eaf2ee6-9897-4d31-ad42-a1900467acfb None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:45.827985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2eaf2ee6-9897-4d31-ad42-a1900467acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:45.859846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:45.860520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:45.860865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:45.861078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:45.861834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:45.862139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:45.863718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:45.887918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 279/568] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Oct 27 13:11:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:45.887918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:45.916256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 290/569] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:45.916256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2ee6-9897-4d31-ad42-a1900467acfb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.081820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.088640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645bd59-7ecf-4053-8ec0-27a4442dcbd8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.117317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2ee6-9897-4d31-ad42-a1900467acfb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.131628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.137376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1b535-7460-43a6-90e7-d6c9dd47db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.138055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1b535-7460-43a6-90e7-d6c9dd47db46 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.138279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1b535-7460-43a6-90e7-d6c9dd47db46 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.138550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1b535-7460-43a6-90e7-d6c9dd47db46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.152259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1b535-7460-43a6-90e7-d6c9dd47db46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.152259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1b535-7460-43a6-90e7-d6c9dd47db46 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.160923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a310ee2-e6f9-4edc-a2be-41496351b3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.161614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a310ee2-e6f9-4edc-a2be-41496351b3af None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.161892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a310ee2-e6f9-4edc-a2be-41496351b3af None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.162163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a310ee2-e6f9-4edc-a2be-41496351b3af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-51c1b535-7460-43a6-90e7-d6c9dd47db46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:46.165148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:46.165990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.166889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 280/570] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:46.168553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414ce0-0365-4379-a3cb-a262ca4a445d None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.168553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414ce0-0365-4379-a3cb-a262ca4a445d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.168553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414ce0-0365-4379-a3cb-a262ca4a445d None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.170016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 281/571] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:46.171800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4414ce0-0365-4379-a3cb-a262ca4a445d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.174300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a310ee2-e6f9-4edc-a2be-41496351b3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.176064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a310ee2-e6f9-4edc-a2be-41496351b3af None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-6a310ee2-e6f9-4edc-a2be-41496351b3af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:46.193048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:46.195268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:46.197397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:46.197397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:46.197397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:46.197397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 291/572] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:46.197397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e91a03-5431-48a1-b313-0efd771c0c00 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.197397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e91a03-5431-48a1-b313-0efd771c0c00 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.197397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e91a03-5431-48a1-b313-0efd771c0c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.220766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-35e91a03-5431-48a1-b313-0efd771c0c00 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:46.221955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-35e91a03-5431-48a1-b313-0efd771c0c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:46.288887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4414ce0-0365-4379-a3cb-a262ca4a445d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.304253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 292/573] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:46.304253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e91a03-5431-48a1-b313-0efd771c0c00 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.340207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.341362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.341984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.342364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.343121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:46.343678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.345317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.362019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 282/574] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:46.363341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.391505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.402678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.418596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e91a03-5431-48a1-b313-0efd771c0c00 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.455040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.455653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.455930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.456199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.456875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:46.457171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.458513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.471882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 293/575] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:46.473620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.491294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c5e01-7548-443e-afa3-d0f8a60d3258 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.538842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.539496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.539775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.540043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.540700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.542136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:46.551713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:46.552722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:46.553528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.556574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 283/576] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:46.558094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.598322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a32c63-1af0-4263-85ad-d3c027986fad None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.639311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.639666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.639939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.640205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.641050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=d1d142bc3a9e4b3badefe713c7405329) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:46.641349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.643329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.646654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.656289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 294/577] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/role_assignments?user.id=d1d142bc3a9e4b3badefe713c7405329&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:46.656289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.674908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.678307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30cb63-1bab-45c3-8f74-b92a75e72571 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.721983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.722923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.723346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.723895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.732492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.734408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:46.747815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:46.748988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:46.750091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.751818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 284/578] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:46.757636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.782830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09866d-b5f0-41af-af59-c49c9d9e0944 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.832641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.833368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.833835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.834114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.834766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:46.835065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.837186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.842520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 295/579] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:46.844265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.882243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb7177-7533-476d-b01f-0ff38953070a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:46.925495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:46.926211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:46.926591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:46.927054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] PATH_INFO: `/v3/services/network` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:46.927740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:46.929351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:46.938631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:46.939769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:46.940667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:46.945320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 285/580] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:46.946004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:46.949984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.959393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:46.978713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8bfe-be20-49cd-a62e-4c8aac154a7c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.018079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.019047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.019525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.020000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.020910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:47.021596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.024491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.037092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 296/581] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:47.038210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:47.060675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696a18b-57bc-457b-8c73-a5ab69b587ae None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.099264 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.100033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.100331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.100608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.101233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:47.101560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.103840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.109833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 286/582] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Oct 27 13:11:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:47.110742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:47.152396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2355a3f-2698-4790-9d5f-c0e1b30adafb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.192638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.193172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.193449 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.193721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.194341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:47.194639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.196753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:47.196865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.205139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 297/583] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:11:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:47.206312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:47.210142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:47.235126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7343f53-0c49-44b0-a249-c73f3d0f4c9d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.272737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.273388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.274263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.274545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/bc93e3b174024520ac01244301f179bc/roles/eb326c809f1543ce8be84f07a0a1cdab` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.290371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=bc93e3b174024520ac01244301f179bc, role_id=eb326c809f1543ce8be84f07a0a1cdab)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.291803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.311817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 287/584] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:47 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/bc93e3b174024520ac01244301f179bc/roles/eb326c809f1543ce8be84f07a0a1cdab => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:47.312739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:47.327884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddf2df-5782-4f82-a4b9-284492305dc6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.370880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.371957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.372498 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.372980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.373990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] RBAC: Adding query filter params (interface=public, service_id=641479c8c2e84b71938b0b94bfd47b0a, region_id=RegionOne) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:47.374652 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.376357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.387665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 298/585] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:11:47 2022] GET /identity/v3/endpoints?service_id=641479c8c2e84b71938b0b94bfd47b0a&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:47.394465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:47.476729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:47.486713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5be64-24df-4f0d-be7f-fc6d54dbf4b3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.516185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:47.519540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26147a7c-852f-4a10-9ada-eb3c73c4932b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.539110 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7201c-b569-4292-8234-172f271a8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.539867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7201c-b569-4292-8234-172f271a8603 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.540403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7201c-b569-4292-8234-172f271a8603 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.540737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7201c-b569-4292-8234-172f271a8603 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.541786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7201c-b569-4292-8234-172f271a8603 None admin] RBAC: Adding query filter params (type=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:47.542146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7201c-b569-4292-8234-172f271a8603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.543606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7201c-b569-4292-8234-172f271a8603 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.551614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 288/586] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:11:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:47.553188 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d95a3-2c47-47c3-9dd2-8d773f2303a7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.553534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d95a3-2c47-47c3-9dd2-8d773f2303a7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.553862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d95a3-2c47-47c3-9dd2-8d773f2303a7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.560038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 289/587] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:47.561079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d95a3-2c47-47c3-9dd2-8d773f2303a7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:47.567872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d96b2-5fc0-4571-850f-86d432cd375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.568539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d96b2-5fc0-4571-850f-86d432cd375d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.568818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d96b2-5fc0-4571-850f-86d432cd375d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.569148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d96b2-5fc0-4571-850f-86d432cd375d None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.569755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d96b2-5fc0-4571-850f-86d432cd375d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.571919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d96b2-5fc0-4571-850f-86d432cd375d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.578625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 299/588] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:11:47 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:47.580182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04cfdb-ce03-4799-9f82-c0d763ac8a9c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.580510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04cfdb-ce03-4799-9f82-c0d763ac8a9c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.580785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04cfdb-ce03-4799-9f82-c0d763ac8a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.609023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ee04cfdb-ce03-4799-9f82-c0d763ac8a9c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:47.611117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee04cfdb-ce03-4799-9f82-c0d763ac8a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:47.719966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d95a3-2c47-47c3-9dd2-8d773f2303a7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.759782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.760764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.761276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.761690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.762542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] RBAC: Adding query filter params (interface=public, service_id=2cff14bff89c4ebc9fa39cf978cdebe3, region_id=RegionOne) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:47.763018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.763194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 300/589] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:47.764913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.779398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 290/590] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:11:47 2022] GET /identity/v3/endpoints?service_id=2cff14bff89c4ebc9fa39cf978cdebe3&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:47.781079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04cfdb-ce03-4799-9f82-c0d763ac8a9c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:47.855375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:47.894216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04cfdb-ce03-4799-9f82-c0d763ac8a9c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:47.928878 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:47.929482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:47.929772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:47.930039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:47.930625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:47.931964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:47.937390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 301/591] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:11:47 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:48.087497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:48.151645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:48.217364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a446cad-ccbf-488d-afb5-943268d4ad10 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:48.269840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:48.270472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:48.270748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:48.271013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:48.281869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:48.287013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:48.287916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:48.289490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 291/592] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:48.294276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:48.413133 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc1a02-e8fc-49da-9f31-7018fb513bed None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:48.444349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:48.452055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:48.452682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:48.452956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:48.453220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:48.453873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:48.454167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:48.455510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:48.463942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 302/593] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:48.473558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:48.583622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a9191-0f5d-4937-9f59-79ba66195ce1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:48.596852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:48.623146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:48.623780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:48.624050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:48.624397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:48.635578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:48.637043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:48.651731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:48.652746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:48.653951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:48.653951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:48.653951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:48.653951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 292/594] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:48.658490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:48.819956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4db3b-1a65-4113-a00a-7c3f6df95922 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:48.865038 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:48.865656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:48.865904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:48.866179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:48.866812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:48.867108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:48.869053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:48.894236 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 303/595] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:48.899575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:49.077870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f1505-d911-4232-8935-1553c2642416 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:49.112010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82269-0a1d-4126-b1b2-e5d808e40ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:49.112671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82269-0a1d-4126-b1b2-e5d808e40ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:49.112918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82269-0a1d-4126-b1b2-e5d808e40ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:49.113178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82269-0a1d-4126-b1b2-e5d808e40ae8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:49.121271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e82269-0a1d-4126-b1b2-e5d808e40ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:49.122879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e82269-0a1d-4126-b1b2-e5d808e40ae8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-74e82269-0a1d-4126-b1b2-e5d808e40ae8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:49.134165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:49.135357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 27 13:11:49.136638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:49.138267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 293/596] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:11:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:49.145302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:49.237423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:49.301095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c411b-ea23-4154-89f9-547e71a73d5b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:49.334788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112f63-4c6c-4783-80fe-2543212af337 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:49.335095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112f63-4c6c-4783-80fe-2543212af337 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:49.335360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112f63-4c6c-4783-80fe-2543212af337 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:49.337660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 294/597] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:49.342576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979683ba-f8f0-44a2-ae18-ac14e20f10fa None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:49.342853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979683ba-f8f0-44a2-ae18-ac14e20f10fa None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:49.343120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979683ba-f8f0-44a2-ae18-ac14e20f10fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:49.353214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:49.353824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:49.354098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:49.354365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:49.355078 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:49.355378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:49.356839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:49.363297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-979683ba-f8f0-44a2-ae18-ac14e20f10fa None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:49.364509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-979683ba-f8f0-44a2-ae18-ac14e20f10fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:49.380973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:49.383205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 304/598] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Oct 27 13:11:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:49.389665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:49.470854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 295/599] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:11:49.498725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fcdca-4ca5-41aa-b6c8-b4c61f09984f None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:49.506253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:49.541818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:49.542773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:49.543264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:49.543756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:49.640588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:49.642367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:49.774823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 305/600] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:11:49.779994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979683ba-f8f0-44a2-ae18-ac14e20f10fa None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:49.790514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:49.805682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:49.942050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979683ba-f8f0-44a2-ae18-ac14e20f10fa None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:49.958956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22081dbe-918b-415d-9713-34eee079fce2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:49.999501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.000241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.000531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.000941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.012698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.014126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:50.030131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:50.031118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:50.032216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:50.032216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:50.032216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:50.032216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 296/601] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:50.036277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.036277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.036277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.036277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.036778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.036778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.041717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:50.045887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:50.055566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:50.057891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:50.060073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 306/602] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Oct 27 13:11:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:50.069180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:50.176204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a24509-5394-4537-94a5-d4d289ace940 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:50.190386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:50.222379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa95c9-83fb-4844-af16-3deb842e9e4e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:50.255553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b449ba0-da5a-4f02-a8af-382a2baf3abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.256171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b449ba0-da5a-4f02-a8af-382a2baf3abe None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.256479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b449ba0-da5a-4f02-a8af-382a2baf3abe None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.256747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b449ba0-da5a-4f02-a8af-382a2baf3abe None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.257394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b449ba0-da5a-4f02-a8af-382a2baf3abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:50.257691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b449ba0-da5a-4f02-a8af-382a2baf3abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.259245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b449ba0-da5a-4f02-a8af-382a2baf3abe None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.263146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.263765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.264036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.264333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.264962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:50.265255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.266648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.273212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 297/603] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:50.273766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 307/604] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:50.274647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce6615-2917-4595-bcc0-0f3dfe030eaa None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.274923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce6615-2917-4595-bcc0-0f3dfe030eaa None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.275191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce6615-2917-4595-bcc0-0f3dfe030eaa None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.279413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:50.281282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 298/605] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:50.282176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce6615-2917-4595-bcc0-0f3dfe030eaa None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:50.345518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:50.385953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce6615-2917-4595-bcc0-0f3dfe030eaa None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:50.394053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81586b7-5e8b-44ff-9d5c-a83bac48b4d9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:50.422426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac494e-4d99-4b63-ab63-7ffd909c0de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.423057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac494e-4d99-4b63-ab63-7ffd909c0de7 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.423333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac494e-4d99-4b63-ab63-7ffd909c0de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.423643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac494e-4d99-4b63-ab63-7ffd909c0de7 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.424278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac494e-4d99-4b63-ab63-7ffd909c0de7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:50.424576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac494e-4d99-4b63-ab63-7ffd909c0de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.425935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac494e-4d99-4b63-ab63-7ffd909c0de7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.431164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e82c38-49cf-489a-9ee4-74d317a06682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.431442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 299/606] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:50.432407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82c38-49cf-489a-9ee4-74d317a06682 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.432667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939955e2-8a68-4c67-ab95-85db248709b2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.432942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939955e2-8a68-4c67-ab95-85db248709b2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.433211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939955e2-8a68-4c67-ab95-85db248709b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.433477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82c38-49cf-489a-9ee4-74d317a06682 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.433916 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82c38-49cf-489a-9ee4-74d317a06682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.442627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:50.450660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e82c38-49cf-489a-9ee4-74d317a06682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.453143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e82c38-49cf-489a-9ee4-74d317a06682 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.459156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-939955e2-8a68-4c67-ab95-85db248709b2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:50.460410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-939955e2-8a68-4c67-ab95-85db248709b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-f4e82c38-49cf-489a-9ee4-74d317a06682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:50.469927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:50.470891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:50.471776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:50.474631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 308/607] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:50.476381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bf389-20c8-43ca-8130-4b758134e0a5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.477426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bf389-20c8-43ca-8130-4b758134e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.477919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bf389-20c8-43ca-8130-4b758134e0a5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.484328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 309/608] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:50.486274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bf389-20c8-43ca-8130-4b758134e0a5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:50.545231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 300/609] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:50.546532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381481e-f8ec-4e23-a739-76ae8b0faac0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.546910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381481e-f8ec-4e23-a739-76ae8b0faac0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.547275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381481e-f8ec-4e23-a739-76ae8b0faac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.569662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4381481e-f8ec-4e23-a739-76ae8b0faac0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:50.571056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4381481e-f8ec-4e23-a739-76ae8b0faac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:50.638100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bf389-20c8-43ca-8130-4b758134e0a5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:50.648098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:50.664910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 301/610] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:50.694979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8651366-cf45-46cb-b746-1a012a319421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.695636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8651366-cf45-46cb-b746-1a012a319421 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.695928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8651366-cf45-46cb-b746-1a012a319421 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.696413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8651366-cf45-46cb-b746-1a012a319421 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.697560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8651366-cf45-46cb-b746-1a012a319421 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:50.697870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8651366-cf45-46cb-b746-1a012a319421 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.699400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8651366-cf45-46cb-b746-1a012a319421 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.718187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 310/611] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:50.724082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381481e-f8ec-4e23-a739-76ae8b0faac0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:50.826613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ef3b-67ef-46e4-9bb8-10996536b75b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.826798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ef3b-67ef-46e4-9bb8-10996536b75b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.827052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ef3b-67ef-46e4-9bb8-10996536b75b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.828663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 311/612] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:50.833573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18adbbf-2ce7-4a1a-8525-3eed2b84777f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.833776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18adbbf-2ce7-4a1a-8525-3eed2b84777f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.833992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18adbbf-2ce7-4a1a-8525-3eed2b84777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.864394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d18adbbf-2ce7-4a1a-8525-3eed2b84777f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:50.865575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d18adbbf-2ce7-4a1a-8525-3eed2b84777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:50.885663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381481e-f8ec-4e23-a739-76ae8b0faac0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:50.933514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:50.935141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:50.935759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:50.936081 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:50.936375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:50.937092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=edb1a2c4a51f46c1be39baf0927eff4a) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:50.937725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:50.939653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:50.956441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 302/613] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/role_assignments?user.id=edb1a2c4a51f46c1be39baf0927eff4a&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:50.958434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:50.965840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 312/614] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:11:50.966573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18adbbf-2ce7-4a1a-8525-3eed2b84777f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.059131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:51.111287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18adbbf-2ce7-4a1a-8525-3eed2b84777f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.151053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66223f4-b694-4aaf-ba9b-0b43de66b9b2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.154142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.154787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.155061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.155323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.155900 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.157323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:51.181923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:51.182977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 27 13:11:51.183814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:51.184569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 313/615] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:51.192051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.231536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.232177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.232497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.232766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.248537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:51.252959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:51.258837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:51.259928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:51.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 303/616] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:11:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:51.268728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.325952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b23629-3d2b-499b-82a7-3bc8c4ac9108 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.361047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.361707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.361925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.362209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.362889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:51.363187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.364576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.371958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 314/617] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:51.376540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.406156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ca023-56c1-4976-8e83-74280e2bcec8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.419707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:51.442890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.443494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.443817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.444085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.444837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:51.445095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.447076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.460179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 304/618] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:51.461124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.513933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f37f2-ea27-4180-83b9-07d7199f3570 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.529561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:51.552000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.552651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.552920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.553181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] PATH_INFO: `/v3/services/network` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.553865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.555837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:51.565207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:51.566196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 27 13:11:51.567033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:51.571818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 315/619] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:51.572803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.580088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae627fc-5a1f-4ee9-9da8-580db7422dd3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.630486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.631087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.631378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.631642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.632290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:51.632591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.634558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.643978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 305/620] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:51.643978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.684046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:51.713489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a9365-5333-4699-930c-3fbcff83c747 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.748958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bc9af-3e5d-4070-9957-32bb2eccecaf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.758445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.759389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.760318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.761080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.774646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.777129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.785159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.785842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.786065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.786335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.786952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] RBAC: Adding query filter params (name=network) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:51.787244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.788712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.798324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 306/621] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:51.798324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:51.802125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:51.803082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:51.803958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:51.803958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:51.803958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:51.807041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 316/622] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:51.808055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.841487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:51.907868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ae658-7656-4e21-a810-b68728a4b807 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.909488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:51.932508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bf113-518e-429d-9cef-7a7bc679b5f8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:51.946264 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.946929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.947207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.947472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.948187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.949821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:51.977884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 307/623] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:11:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:51.977884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:51.989388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:51.990722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:51.991210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:51.991697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:51.993340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] RBAC: Adding query filter params (type=network) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:51.994355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:51.996328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:52.009458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 317/624] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:52.018207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:52.125620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:52.139178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009e69-ca76-43a8-9c80-7066dded7c9d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:52.172216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef9a7c-19c4-47c6-afb6-1908be88bcc4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:52.218296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:52.219315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:52.219804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:52.222298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:52.223245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:52.227155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:52.250731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:52.251371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:52.251612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:52.251917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:52.252628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:52.253086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:52.254610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:52.265563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 318/625] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:11:52 2022] POST /identity/v3/endpoints => generated 337 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:11:52.274792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 308/626] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:52.280514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:52.405400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328d08b-39a0-4af2-8068-6396b2db4cfe None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:52.444106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:52.445178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:52.446188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:52.446678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:52.455110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:52.457000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:52.460267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:52.472861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:52.474027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:11:52.475150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:52.477601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 319/627] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:11:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:52.483948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:52.503431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:52.600246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afddc9f-3cc7-4115-bc72-c7e62e4ec10e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:52.639651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fde4ac-8c86-4c1e-8a7a-199e59ce903e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:52.640280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fde4ac-8c86-4c1e-8a7a-199e59ce903e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:52.640563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fde4ac-8c86-4c1e-8a7a-199e59ce903e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:52.640831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fde4ac-8c86-4c1e-8a7a-199e59ce903e None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:52.641552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fde4ac-8c86-4c1e-8a7a-199e59ce903e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:52.642115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fde4ac-8c86-4c1e-8a7a-199e59ce903e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:52.643522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fde4ac-8c86-4c1e-8a7a-199e59ce903e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:52.666630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 309/628] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Oct 27 13:11:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:52.672199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:52.795326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ea7bc-252a-4137-96a6-a34d269ed66b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:52.826236 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72d719-7386-4d8d-9ea3-720646e6c901 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:52.826558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72d719-7386-4d8d-9ea3-720646e6c901 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:52.826900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72d719-7386-4d8d-9ea3-720646e6c901 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:52.832420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 310/629] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:52.833049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:52.833652 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:52.833757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1165f-094c-4a86-aa62-fd5325b314fc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:52.833931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:52.834026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1165f-094c-4a86-aa62-fd5325b314fc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:52.834171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:52.834270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1165f-094c-4a86-aa62-fd5325b314fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:52.853039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e2a1165f-094c-4a86-aa62-fd5325b314fc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:52.854217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e2a1165f-094c-4a86-aa62-fd5325b314fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:52.921071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:52.924715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:52.926985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:52.936310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 311/630] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:11:53.039146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 320/631] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:11:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:11:53.045035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1165f-094c-4a86-aa62-fd5325b314fc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:53.054100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:53.150020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1165f-094c-4a86-aa62-fd5325b314fc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:53.185454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:53.186138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.186413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.186677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.197203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:53.198650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:53.213499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:53.214455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:53.215335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:53.215335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:53.215335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:53.215335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 312/632] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:53.220358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:53.226057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:53.293560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:53.329097 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:53.332643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f475302-e728-4297-a6ab-664244d21000 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:53.354709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aec21-6580-4d5c-9c46-3a36801ffbcd None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:53.371875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:53.372812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.373230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.373636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.374483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:53.374942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:53.376514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:53.391604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 321/633] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:53.396374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab9d6c-5ef0-46aa-a48d-8ce734d177ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:53.397415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:53.398460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab9d6c-5ef0-46aa-a48d-8ce734d177ed None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.398734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab9d6c-5ef0-46aa-a48d-8ce734d177ed None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.399047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab9d6c-5ef0-46aa-a48d-8ce734d177ed None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.399672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab9d6c-5ef0-46aa-a48d-8ce734d177ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:53.401148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab9d6c-5ef0-46aa-a48d-8ce734d177ed None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:53.420326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 313/634] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:11:53 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:53.421692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3d1ef-6d83-484e-8565-e601882db9ef None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.421973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3d1ef-6d83-484e-8565-e601882db9ef None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.422243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3d1ef-6d83-484e-8565-e601882db9ef None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.423759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 314/635] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:53.431031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074106-7ad8-44e6-888d-f7d6841892e1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.431359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074106-7ad8-44e6-888d-f7d6841892e1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.431566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074106-7ad8-44e6-888d-f7d6841892e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.454512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-af074106-7ad8-44e6-888d-f7d6841892e1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:53.456402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-af074106-7ad8-44e6-888d-f7d6841892e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:53.524844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff801e-ebda-4e86-b34f-d0c75a780cec None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:53.563939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:53.565067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.565568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.565807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 315/636] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:53 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:11:53.566288 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.574324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:53.576158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:53.595363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:53.596340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:53.597526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 322/637] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:53.604582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074106-7ad8-44e6-888d-f7d6841892e1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:53.654525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:53.710476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074106-7ad8-44e6-888d-f7d6841892e1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:53.745506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:53.746109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.746384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.746655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.747336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:53.747636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:53.749072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:53.760870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 316/638] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:53.769317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:53.792357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:53.871150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88620c-2e16-4024-884b-ef12193775b9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:53.878329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:53.912428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:53.913045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:53.913618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:53.913925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:53.914875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] RBAC: Adding query filter params (role.id=eb326c809f1543ce8be84f07a0a1cdab, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=bc93e3b174024520ac01244301f179bc) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:53.915229 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:53.916885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:53.924755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 323/639] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:11:53 2022] GET /identity/v3/role_assignments?user.id=bc93e3b174024520ac01244301f179bc&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=eb326c809f1543ce8be84f07a0a1cdab => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:54.014550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0e62-4459-4dd0-b6fc-4be736bc2c63 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:54.017612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:54.053700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:54.054349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:54.055189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:54.055461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:54.056049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:54.057476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:54.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:54.068532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:54.069385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 317/640] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:11:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:54.075010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:54.168494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:54.185553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b29ecb-5736-4b43-8b56-6b0f5eba84f3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:54.218991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb9b2a-94c7-4525-8fed-e55015e07449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:54.219600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb9b2a-94c7-4525-8fed-e55015e07449 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:54.219869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb9b2a-94c7-4525-8fed-e55015e07449 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:54.220130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb9b2a-94c7-4525-8fed-e55015e07449 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:54.220799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb9b2a-94c7-4525-8fed-e55015e07449 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:54.221089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb9b2a-94c7-4525-8fed-e55015e07449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:54.222440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb9b2a-94c7-4525-8fed-e55015e07449 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:54.227914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 324/641] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:11:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:54.233529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:54.339993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2db51-309c-423e-806d-3de32931d0c0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:54.374692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21772778-069b-4960-afdc-b583a996402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:54.375621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772778-069b-4960-afdc-b583a996402c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:54.376068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772778-069b-4960-afdc-b583a996402c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:54.376538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772778-069b-4960-afdc-b583a996402c None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:54.377330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772778-069b-4960-afdc-b583a996402c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:54.377797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772778-069b-4960-afdc-b583a996402c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:54.379309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772778-069b-4960-afdc-b583a996402c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:54.384559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 318/642] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:11:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:54.580901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04943ee-e63a-4050-84d0-55335793e0c7 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:54.581457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04943ee-e63a-4050-84d0-55335793e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:54.581901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04943ee-e63a-4050-84d0-55335793e0c7 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:54.583720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 325/643] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:54.589984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b924-f4a6-4eea-bbf9-402ce8d09165 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:54.590271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b924-f4a6-4eea-bbf9-402ce8d09165 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:54.590551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b924-f4a6-4eea-bbf9-402ce8d09165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:54.610717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fda4b924-f4a6-4eea-bbf9-402ce8d09165 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:54.617484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fda4b924-f4a6-4eea-bbf9-402ce8d09165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:54.644070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:54.688941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 319/644] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:54 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:54.956336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04943ee-e63a-4050-84d0-55335793e0c7 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:55.055869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04943ee-e63a-4050-84d0-55335793e0c7 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:55.089477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:55.090238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:55.090425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:55.090696 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:55.091267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:55.092640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:55.102083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:55.103043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:55.103864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 326/645] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:11:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:55.109306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4b924-f4a6-4eea-bbf9-402ce8d09165 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:55.215445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4b924-f4a6-4eea-bbf9-402ce8d09165 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:55.251395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d83778-6495-46ea-831b-a5ada462c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:55.252012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d83778-6495-46ea-831b-a5ada462c7cd None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:55.252319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d83778-6495-46ea-831b-a5ada462c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:55.252594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d83778-6495-46ea-831b-a5ada462c7cd None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:55.253221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d83778-6495-46ea-831b-a5ada462c7cd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:55.253522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d83778-6495-46ea-831b-a5ada462c7cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:55.254970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d83778-6495-46ea-831b-a5ada462c7cd None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:55.260359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 320/646] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:11:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:55.265539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:55.282261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:55.380884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83de57-d648-44ae-a364-d6731d8066dd None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:55.416045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21e4fb-74eb-4244-b862-33f1643af646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:55.416686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21e4fb-74eb-4244-b862-33f1643af646 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:55.416962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21e4fb-74eb-4244-b862-33f1643af646 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:55.417233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21e4fb-74eb-4244-b862-33f1643af646 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:55.417853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21e4fb-74eb-4244-b862-33f1643af646 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:55.418150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21e4fb-74eb-4244-b862-33f1643af646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:55.419497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21e4fb-74eb-4244-b862-33f1643af646 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:55.424414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 327/647] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:11:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:55.731686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61fb55-a93c-4294-9d76-2d460fb20b05 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:55.731686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61fb55-a93c-4294-9d76-2d460fb20b05 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:55.731686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61fb55-a93c-4294-9d76-2d460fb20b05 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:55.732796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 321/648] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:55.740276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05543f97-8541-423e-ade0-13b267d3e89b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:55.740276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05543f97-8541-423e-ade0-13b267d3e89b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:55.740276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05543f97-8541-423e-ade0-13b267d3e89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:55.757239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-05543f97-8541-423e-ade0-13b267d3e89b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:55.759253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-05543f97-8541-423e-ade0-13b267d3e89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:55.826542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 328/649] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:55 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:56.096571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61fb55-a93c-4294-9d76-2d460fb20b05 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:56.129056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:56.208634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61fb55-a93c-4294-9d76-2d460fb20b05 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:56.239250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc66c-9045-45eb-a34b-1085cdd263ff None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.239764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc66c-9045-45eb-a34b-1085cdd263ff None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.239764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc66c-9045-45eb-a34b-1085cdd263ff None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.241436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 329/650] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:56.247601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf701aa-ab0a-4bbe-a74f-fa7cf2dd8fd0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.248262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf701aa-ab0a-4bbe-a74f-fa7cf2dd8fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.248834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf701aa-ab0a-4bbe-a74f-fa7cf2dd8fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.250743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:56.251410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.251688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.251961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.252588 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:56.253935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:56.263874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:56.264989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:56.269347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 322/651] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Oct 27 13:11:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:56.271796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:56.272583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:56.278135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-fbf701aa-ab0a-4bbe-a74f-fa7cf2dd8fd0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:56.279540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fbf701aa-ab0a-4bbe-a74f-fa7cf2dd8fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:56.365260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 330/652] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:56 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:56.384620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:56.388782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd3cb0-72db-45ca-aac6-5aad38bb0638 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:56.425838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:56.426677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.427123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.427571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.428378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:56.428827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:56.430973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:56.436058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 323/653] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:11:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:56.445369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf701aa-ab0a-4bbe-a74f-fa7cf2dd8fd0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:56.548426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf701aa-ab0a-4bbe-a74f-fa7cf2dd8fd0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:56.587385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:56.588009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.588304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.588569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.589200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:56.589494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:56.590876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:56.596740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:56.600591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 331/654] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:11:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:56.602936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:56.674809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:56.717201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf4dfb-bde3-4c21-999f-70edf8db1adf None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:56.750829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6c0c7-376f-4785-90a5-d476e6cb677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:56.751426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6c0c7-376f-4785-90a5-d476e6cb677d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.752117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6c0c7-376f-4785-90a5-d476e6cb677d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.752117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6c0c7-376f-4785-90a5-d476e6cb677d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.752688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6c0c7-376f-4785-90a5-d476e6cb677d None admin] RBAC: Adding query filter params (interface=public, service_id=5118b977c93340e8b82507997de66ea1, region_id=RegionOne) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:56.752991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6c0c7-376f-4785-90a5-d476e6cb677d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:56.754326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6c0c7-376f-4785-90a5-d476e6cb677d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:56.760007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 324/655] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:11:56 2022] GET /identity/v3/endpoints?service_id=5118b977c93340e8b82507997de66ea1&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:56.762906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c0290-823c-4d9f-b614-e6daf511575a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.763435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c0290-823c-4d9f-b614-e6daf511575a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.763933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c0290-823c-4d9f-b614-e6daf511575a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.766273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 325/656] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:56.767738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c0290-823c-4d9f-b614-e6daf511575a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:56.788925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fe4b0-2b53-49bb-be48-a135759471b4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:56.793360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:56.825153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ab38c-0042-4876-9b4e-7d137d5f990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:56.825767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ab38c-0042-4876-9b4e-7d137d5f990b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.826032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ab38c-0042-4876-9b4e-7d137d5f990b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.826298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ab38c-0042-4876-9b4e-7d137d5f990b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.837586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ab38c-0042-4876-9b4e-7d137d5f990b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-642ab38c-0042-4876-9b4e-7d137d5f990b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:11:56.842985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:56.843887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:56.844757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 332/657] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Oct 27 13:11:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:11:56.845448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c5b7c-1fad-4552-8678-7e3dc129462c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.845448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c5b7c-1fad-4552-8678-7e3dc129462c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.846215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c5b7c-1fad-4552-8678-7e3dc129462c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.863675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3a0c5b7c-1fad-4552-8678-7e3dc129462c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:56.864850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3a0c5b7c-1fad-4552-8678-7e3dc129462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:56.887016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c0290-823c-4d9f-b614-e6daf511575a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:56.922423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:56.923011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:56.923131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:56.923342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:56.923618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:56.924211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:56.925961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:56.931859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 326/658] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:11:56 2022] GET /identity/v3/services => generated 1029 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:11:56.932762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:56.943133 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 333/659] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:56 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:57.038438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:57.059145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b7d5-1d5b-4f89-b85f-4cac4c2b5768 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:57.098380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:57.099844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.100518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.100981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.102412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:57.102902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:57.104466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:57.113782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 327/660] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Oct 27 13:11:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:57.119843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c5b7c-1fad-4552-8678-7e3dc129462c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:57.222253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c5b7c-1fad-4552-8678-7e3dc129462c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:57.228694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:57.260209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:57.261615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.262110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.262548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.273754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:57.275594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:57.291977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:57.292941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:57.293801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:57.293801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:57.293801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:57.296013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 334/661] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:57.301671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:57.337956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3541c-e7db-409c-a04f-8011d4c5bf3b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:57.357170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:57.375259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:57.376005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.376411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.376770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.377517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:57.379164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:57.400430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 328/662] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:11:57 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:11:57.431484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f466e-bddd-4843-8bcc-25a2ddc8f01c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:57.479737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8f2d-c9d1-454a-889a-c1fe7c01ade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:57.480846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8f2d-c9d1-454a-889a-c1fe7c01ade9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.481339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8f2d-c9d1-454a-889a-c1fe7c01ade9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.481861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8f2d-c9d1-454a-889a-c1fe7c01ade9 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.482830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8f2d-c9d1-454a-889a-c1fe7c01ade9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:57.483406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8f2d-c9d1-454a-889a-c1fe7c01ade9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:57.485283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8f2d-c9d1-454a-889a-c1fe7c01ade9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:57.499930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 335/663] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:57.506210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:57.510743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:57.613147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba8bc4-c0df-451f-af6f-69d6c259e84d None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.613424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba8bc4-c0df-451f-af6f-69d6c259e84d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.613693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba8bc4-c0df-451f-af6f-69d6c259e84d None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.615268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 336/664] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:11:57.621565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f48b2-16d9-47f8-947e-17ef0e70792a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.621958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f48b2-16d9-47f8-947e-17ef0e70792a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.622297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f48b2-16d9-47f8-947e-17ef0e70792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.633930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67093ff0-8d8f-4255-819c-2b52746ef9b6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:57.644722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-591f48b2-16d9-47f8-947e-17ef0e70792a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:57.645906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-591f48b2-16d9-47f8-947e-17ef0e70792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:57.669967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:57.670701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.670977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.671244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.679538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:57.681152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:11:57.693115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:57.694067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:57.695715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 329/665] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:11:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:57.700595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:57.740793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 337/666] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:57 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:11:57.765209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:57.815061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c5457-5b06-4880-bfd7-9d1f37f7062a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:57.856661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:57.857323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:57.857593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:57.857858 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:57.858653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:57.858950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:57.860369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:57.878682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 330/667] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Oct 27 13:11:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:57.884901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f48b2-16d9-47f8-947e-17ef0e70792a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:57.915662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:57.989214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f48b2-16d9-47f8-947e-17ef0e70792a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:58.024727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:58.025668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:58.026164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:58.026649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:58.032508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:58.037514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:58.039207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:11:58.054998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:58.055954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:11:58.056869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:11:58.056869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:11:58.056869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:58.059071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 338/668] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:58.064062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:58.143693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad2937-0ca7-4ba0-a3c4-6d94801ec384 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:58.150952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:58.170298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:58.172949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf33d93-bf76-49ca-9a43-20d37faff86e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:58.183881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:58.184415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:58.184689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:58.184955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:58.185601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:58.187121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:58.208766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 331/669] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:11:58 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:11:58.218505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:58.219537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:58.220101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:58.220641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:58.221538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:58.222146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:58.223762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:58.238245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 339/670] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:11:58.245207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:58.365084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7046ca-8322-4bfb-a5dc-1148a865f247 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:58.403932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:58.404582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:58.404853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:58.405115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:58.411882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:58.413324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:11:58.424505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:58.425348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:11:58.426230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 332/671] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:11:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:58.431963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:58.533763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf189f2-5aac-413e-83ef-5a7c65acb609 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:58.568871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6108965-fc04-4c09-ba29-d71a89ff8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:58.569477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108965-fc04-4c09-ba29-d71a89ff8bbe None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:58.569750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108965-fc04-4c09-ba29-d71a89ff8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:58.570135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108965-fc04-4c09-ba29-d71a89ff8bbe None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:58.570815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6108965-fc04-4c09-ba29-d71a89ff8bbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:58.571109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6108965-fc04-4c09-ba29-d71a89ff8bbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:58.572543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6108965-fc04-4c09-ba29-d71a89ff8bbe None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:58.575056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:58.586297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 340/672] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:11:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:58.593499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:58.642068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:58.701382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cb367-5526-4acc-8b80-934c61994cd6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:58.736317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82664d6f-86c1-4e89-b2a0-0a6c1a738ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:58.737351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82664d6f-86c1-4e89-b2a0-0a6c1a738ee1 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:58.737829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82664d6f-86c1-4e89-b2a0-0a6c1a738ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:58.738314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82664d6f-86c1-4e89-b2a0-0a6c1a738ee1 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/bc93e3b174024520ac01244301f179bc/roles/60e2466e171e4daca6ebf84d6504372f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:58.740732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82664d6f-86c1-4e89-b2a0-0a6c1a738ee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=bc93e3b174024520ac01244301f179bc, role_id=60e2466e171e4daca6ebf84d6504372f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:58.742937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82664d6f-86c1-4e89-b2a0-0a6c1a738ee1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:58.760666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 333/673] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:11:58 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/bc93e3b174024520ac01244301f179bc/roles/60e2466e171e4daca6ebf84d6504372f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:11:59.205392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ee15-d025-4159-b45b-18f381ec10ba None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:59.205771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ee15-d025-4159-b45b-18f381ec10ba None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:59.206015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ee15-d025-4159-b45b-18f381ec10ba None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:59.207478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 341/674] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:59.213756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e584e-a5b4-472e-b1d2-52620d4137e4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:59.214639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e584e-a5b4-472e-b1d2-52620d4137e4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:59.215105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e584e-a5b4-472e-b1d2-52620d4137e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:59.234656 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-211e584e-a5b4-472e-b1d2-52620d4137e4 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:59.236273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-211e584e-a5b4-472e-b1d2-52620d4137e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:59.304657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 334/675] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:11:59.583219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522ee15-d025-4159-b45b-18f381ec10ba None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:59.692512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba036a-ce2e-47ef-a28d-cd4d9fcaffda None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:59.693143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba036a-ce2e-47ef-a28d-cd4d9fcaffda None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:59.693641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba036a-ce2e-47ef-a28d-cd4d9fcaffda None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:59.695876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522ee15-d025-4159-b45b-18f381ec10ba None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:59.700026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 335/676] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:11:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:11:59.701687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06623f7b-7949-4bd6-b2d3-a722cdb06230 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:59.702190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06623f7b-7949-4bd6-b2d3-a722cdb06230 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:59.702661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06623f7b-7949-4bd6-b2d3-a722cdb06230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:59.721048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-06623f7b-7949-4bd6-b2d3-a722cdb06230 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:11:59.722606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-06623f7b-7949-4bd6-b2d3-a722cdb06230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:11:59.731191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:59.731821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:59.732099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:59.732404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:59.732975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:59.734427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:11:59.744302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:11:59.745234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:11:59.746141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 342/677] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Oct 27 13:11:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:11:59.752790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:11:59.755973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:59.800143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 336/678] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:11:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:11:59.822191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:11:59.861991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f420fb-cee8-45d0-aa86-a63bd00fe942 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:11:59.895406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:11:59.896477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:11:59.897010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:11:59.897470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:11:59.898297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:11:59.898836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:11:59.900424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:11:59.905712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 343/679] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:11:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:11:59.911525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06623f7b-7949-4bd6-b2d3-a722cdb06230 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:00.009841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06623f7b-7949-4bd6-b2d3-a722cdb06230 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:00.044591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:00.045225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.045507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.045776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.046408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:00.046708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:00.048063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:00.053484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 337/680] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:11:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:00.059714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:00.090936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:00.120136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:00.181360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293f907-3fd3-40f2-b1f1-2a323b5306aa None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:00.198952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6d841-1e03-4218-bb5f-e761e6722402 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:00.210631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:00.224597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:00.225365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.225654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.225924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.227626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:00.229270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:00.238577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:00.239803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.240341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.240821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.241460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:00.243374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:00.253033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:00.254064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:00.254886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:00.258755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 344/682] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:12:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:00.258873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 338/682] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:12:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:00.264308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:00.371575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:00.384132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0b7fe-12e5-4d89-b234-183017785537 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:00.433369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:00.433369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.433369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.433760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.434245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:00.434537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:00.435864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:00.442315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 339/683] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:00.448117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:00.565979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541eedf-28c0-41b7-94c0-31fe9b132959 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:00.608053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0bfbe-9255-4438-8793-c8f38fae58c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:00.609503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0bfbe-9255-4438-8793-c8f38fae58c2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.610011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0bfbe-9255-4438-8793-c8f38fae58c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.610491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0bfbe-9255-4438-8793-c8f38fae58c2 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.611319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0bfbe-9255-4438-8793-c8f38fae58c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:00.612367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0bfbe-9255-4438-8793-c8f38fae58c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:00.613952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0bfbe-9255-4438-8793-c8f38fae58c2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:00.622113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 345/684] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:00.627675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:00.682734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1664e4-3a53-4fd6-a30a-532da8c73677 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.683420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1664e4-3a53-4fd6-a30a-532da8c73677 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.683928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1664e4-3a53-4fd6-a30a-532da8c73677 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.686778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 346/685] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:12:00.691231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df1bf3-c676-44e1-b5a9-5cff91270014 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.692119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df1bf3-c676-44e1-b5a9-5cff91270014 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.692949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df1bf3-c676-44e1-b5a9-5cff91270014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.715709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-86df1bf3-c676-44e1-b5a9-5cff91270014 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:00.717928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-86df1bf3-c676-44e1-b5a9-5cff91270014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:00.738241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2987c6d-b371-432b-8211-a3d060f896d4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:00.770847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:00.774707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:00.775365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.775637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.775952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.776692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] RBAC: Adding query filter params (interface=public, service_id=951f3a243b804d408ad6c83c1da4d799, region_id=RegionOne) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:00.776999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:00.778404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:00.788849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 340/686] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:12:00 2022] GET /identity/v3/endpoints?service_id=951f3a243b804d408ad6c83c1da4d799&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:00.790466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:00.807033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:00.808105 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 347/687] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:00.908955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495a238-de14-4ab9-a3ea-89311e8f1ce5 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:00.944690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de65bd-6346-48be-9e1a-13779ff60e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:00.946126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de65bd-6346-48be-9e1a-13779ff60e2c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.946548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de65bd-6346-48be-9e1a-13779ff60e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.946975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de65bd-6346-48be-9e1a-13779ff60e2c None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.947714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de65bd-6346-48be-9e1a-13779ff60e2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:00.949237 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de65bd-6346-48be-9e1a-13779ff60e2c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:00.956156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 341/688] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:12:00 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:12:00.989090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d647c01-faf2-4f22-9469-9619a1683feb None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.989379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d647c01-faf2-4f22-9469-9619a1683feb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.989660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d647c01-faf2-4f22-9469-9619a1683feb None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:00.991308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 348/689] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:00.997113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da33e5-f729-492a-98aa-e25e8132e638 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:00.997396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da33e5-f729-492a-98aa-e25e8132e638 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:00.997674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da33e5-f729-492a-98aa-e25e8132e638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.018309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-10da33e5-f729-492a-98aa-e25e8132e638 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:01.019612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-10da33e5-f729-492a-98aa-e25e8132e638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:01.100714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 342/690] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:01.118836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d647c01-faf2-4f22-9469-9619a1683feb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.226316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d647c01-faf2-4f22-9469-9619a1683feb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.261321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.261953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.262224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.262490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.263062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:01.264448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:01.273796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:01.274844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:01.275670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 349/691] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:01.283208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da33e5-f729-492a-98aa-e25e8132e638 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.295521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:01.387147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.392412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da33e5-f729-492a-98aa-e25e8132e638 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.430404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.431019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.431298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.431570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.432196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:01.432577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:01.433995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:01.435209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:01.440029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 343/692] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:01.446266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.504603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45a4e6-b501-4999-8190-c3c2c30244b7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.522151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:01.558104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb14f7-4a7e-426e-9be9-f7199fe89226 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.560184 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.561186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.561637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.562076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.573362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:01.578985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:01.579877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:12:01.580867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:01.583169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 350/693] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:01.592093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.597375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.597993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.598271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.598540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.599606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:01.599906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:01.602270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:01.609220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 344/694] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:01.614989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.704624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:01.707213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba6d5-ae9f-4287-8a93-972d09faf590 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.719796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39794c86-41a0-463d-8384-68e19a6fa5d1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.747334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.747968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.748333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.748607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.749359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:01.749662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:01.751269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:01.756864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.757439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.757721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.757995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.758752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] RBAC: Adding query filter params (interface=public, service_id=6a5960643a9a45a18b36b004dd75ddfe, region_id=RegionOne) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:01.759064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:01.760439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 351/695] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:01.760582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:01.763226 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:01.767026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 345/696] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/endpoints?service_id=6a5960643a9a45a18b36b004dd75ddfe&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:01.767448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.772894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:01.884223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69370c-4923-4a9d-9b06-751689a22b56 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.904410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a4e19-33f3-4315-a83a-35dfe6ec6d96 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:01.927517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.928615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.929000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.929894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.943004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:01.943624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:01.943924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:01.944353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:01.945065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:01.946492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:01.946614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:01.948808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:01.961105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 346/697] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:12:01.965254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:01.966265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:12:01.967175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:12:01.967175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:01.967175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:01.967175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 352/698] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:01.972382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:02.053695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:02.109934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7add3cf-534d-4c7b-9f41-5de69d18f2f1 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:02.150113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:02.150724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:02.151003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:02.151275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:02.151914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:02.152214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:02.153603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:02.166326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 347/699] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:02.171922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:02.194367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:02.285484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6ad5d-ad66-4c08-b4a0-c2f143a70bb9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:02.321557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:02.322517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:02.322976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:02.323440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:02.331354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:02.333015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:12:02.345752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:02.346731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 27 13:12:02.347664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:02.349832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 353/700] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:12:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:02.356432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:02.462411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac862177-cf48-43c1-a757-ebe9fef89a2e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:02.488924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:02.500444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec232b67-a266-4685-8fd9-bdce6891e185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:02.501060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec232b67-a266-4685-8fd9-bdce6891e185 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:02.501386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec232b67-a266-4685-8fd9-bdce6891e185 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:02.501665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec232b67-a266-4685-8fd9-bdce6891e185 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:02.502781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec232b67-a266-4685-8fd9-bdce6891e185 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:02.503316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec232b67-a266-4685-8fd9-bdce6891e185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:02.505136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec232b67-a266-4685-8fd9-bdce6891e185 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:02.523365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 348/701] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Oct 27 13:12:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:02.528752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:02.610218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:02.645731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d95c20-b51e-4fdb-85c2-6baacebf6aca None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:02.660103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9299d2-bdb0-45c1-9df4-0371a26a366f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:02.660433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9299d2-bdb0-45c1-9df4-0371a26a366f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:02.660713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9299d2-bdb0-45c1-9df4-0371a26a366f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:02.662306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 349/702] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:02.667799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745de02-1944-431b-824d-73e31871da2c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:02.668083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745de02-1944-431b-824d-73e31871da2c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:02.668417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745de02-1944-431b-824d-73e31871da2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:02.685327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:02.686275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:02.686746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:02.687234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:02.688345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2745de02-1944-431b-824d-73e31871da2c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:02.689592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2745de02-1944-431b-824d-73e31871da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:02.758354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:02.760487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:02.768758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 350/703] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:02.831184 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:02.864200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 354/704] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:12:02 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:12:02.870026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745de02-1944-431b-824d-73e31871da2c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:02.975479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745de02-1944-431b-824d-73e31871da2c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:02.997024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:03.011479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.012123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.012808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.013088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.023716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.025165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:12:03.040375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:03.041307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:12:03.042774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:12:03.042774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:03.042774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:03.042774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 351/705] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:03.047021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.060263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.161729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9965b6b-1ed3-413e-9731-91ca5b040c4e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.174266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d99f9-ed41-4c66-afdc-c8a239a8104e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.202037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.203371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.203929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.204407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.205208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:03.205630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.207299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.215929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.217018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.217493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.217938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.218841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.220223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.222804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 355/706] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:03.229469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:03.234843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:03.235877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:03.236748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 352/707] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:03.242511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.268106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:03.282516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:03.347521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac7ca8-f796-4763-9e73-c5a6ef0bde3b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.369505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01993-2aea-42bb-b4ad-2f5e49fce619 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.383562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914ec7b-853d-4ee8-89c4-bd9d88e8b812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.384267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914ec7b-853d-4ee8-89c4-bd9d88e8b812 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.384482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914ec7b-853d-4ee8-89c4-bd9d88e8b812 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.384750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914ec7b-853d-4ee8-89c4-bd9d88e8b812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.391407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914ec7b-853d-4ee8-89c4-bd9d88e8b812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.393137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914ec7b-853d-4ee8-89c4-bd9d88e8b812 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5914ec7b-853d-4ee8-89c4-bd9d88e8b812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:12:03.404611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:03.405469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:03.406367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:03.406788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8c24b-4423-43fa-99bc-32577c30e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.406788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c24b-4423-43fa-99bc-32577c30e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.406937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 356/708] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:03.407568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e01a-8ce5-407b-9d84-401b42926e01 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.407568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e01a-8ce5-407b-9d84-401b42926e01 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.407732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e01a-8ce5-407b-9d84-401b42926e01 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.408587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c24b-4423-43fa-99bc-32577c30e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.412952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 357/709] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:03.412952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e01a-8ce5-407b-9d84-401b42926e01 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.413140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8c24b-4423-43fa-99bc-32577c30e3a9 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.413370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8c24b-4423-43fa-99bc-32577c30e3a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:03.414276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8c24b-4423-43fa-99bc-32577c30e3a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.415640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8c24b-4423-43fa-99bc-32577c30e3a9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.424285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 353/710] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:03.424285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb92e3-a672-43c9-a657-235c559e3d71 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.424285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb92e3-a672-43c9-a657-235c559e3d71 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.424285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb92e3-a672-43c9-a657-235c559e3d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.439965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-41cb92e3-a672-43c9-a657-235c559e3d71 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:03.441148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-41cb92e3-a672-43c9-a657-235c559e3d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:03.507609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:03.524255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 354/711] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:03.524255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb92e3-a672-43c9-a657-235c559e3d71 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.526164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e01a-8ce5-407b-9d84-401b42926e01 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.550148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:03.564037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.564932 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.565507 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.565948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.566909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:03.567377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.569122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.581691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 358/712] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:03.588560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.636661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb92e3-a672-43c9-a657-235c559e3d71 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.679610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.680355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.680729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.681090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.681809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:03.682203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.683644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.696189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 355/713] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:03.716073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5a584-a409-4bf6-934f-d8f0e4fc0e4b None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.755115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.755735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.756007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.756298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.757024 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] RBAC: Adding query filter params (role.id=60e2466e171e4daca6ebf84d6504372f, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=bc93e3b174024520ac01244301f179bc) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:03.757378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.758807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.765922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 359/714] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/role_assignments?user.id=bc93e3b174024520ac01244301f179bc&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=60e2466e171e4daca6ebf84d6504372f => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:03.820539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:03.922099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:03.940040 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd31af-095c-4928-8772-81917ae5d865 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:03.985623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:03.988252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:03.988252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:03.988252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:03.988252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:03.988765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:03.998444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:03.999470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:04.000343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 356/715] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:12:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:04.013975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:04.036383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:04.144734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54edffc-a411-4c13-8d92-ca2cee8e23c7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:04.181895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec4d9-ec48-44f4-9837-0956b2155222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:04.181895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec4d9-ec48-44f4-9837-0956b2155222 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:04.181895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec4d9-ec48-44f4-9837-0956b2155222 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:04.181895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec4d9-ec48-44f4-9837-0956b2155222 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:04.182299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aec4d9-ec48-44f4-9837-0956b2155222 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:04.182599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aec4d9-ec48-44f4-9837-0956b2155222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:04.183953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aec4d9-ec48-44f4-9837-0956b2155222 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:04.189320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 360/716] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:04.197655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:04.316205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea92da-6429-431e-8a63-34b855dd1e97 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:04.350561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70490f5-dad8-4b9b-b785-1e9d93c47cb9 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:04.350894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70490f5-dad8-4b9b-b785-1e9d93c47cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:04.351679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70490f5-dad8-4b9b-b785-1e9d93c47cb9 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:04.353304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 361/717] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:12:04.356474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:04.357149 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:04.357423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:04.357691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:04.358371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:04.358671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:04.360015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:04.366190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 357/718] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:04.367155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c88a-d60d-44b0-9fe6-b8d630591e9a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:04.367435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c88a-d60d-44b0-9fe6-b8d630591e9a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:04.367707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c88a-d60d-44b0-9fe6-b8d630591e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:04.374667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:04.402352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-05f4c88a-d60d-44b0-9fe6-b8d630591e9a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:04.403521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-05f4c88a-d60d-44b0-9fe6-b8d630591e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:04.406805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:04.513541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a93df8-74ca-470c-8fc3-76f3f006b193 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:04.516369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 362/719] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:04.550597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:04.551629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:04.552130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:04.552632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:04.553496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:04.555168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:04.577663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 358/720] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:12:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:04.801565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c88a-d60d-44b0-9fe6-b8d630591e9a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:04.813373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:04.902657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c88a-d60d-44b0-9fe6-b8d630591e9a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:04.942026 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:04.942655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:04.942925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:04.943187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:04.943749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:04.945749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:04.954379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:04.955365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:04.956218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 363/721] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:12:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:04.961562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:05.038340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:05.078333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e0403-fd99-416d-ae5e-b85451d76ec3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:05.112372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:05.112979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:05.113282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:05.113511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:05.114133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:05.114424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:05.115765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:05.121565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 359/722] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:12:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:05.127094 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:05.197957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:05.229798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae3013-9736-4c25-9b86-7cc7b2450fe7 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:05.264093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb6e19-5c12-45c4-8834-59b38b867fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:05.264725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb6e19-5c12-45c4-8834-59b38b867fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:05.264995 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb6e19-5c12-45c4-8834-59b38b867fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:05.265264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb6e19-5c12-45c4-8834-59b38b867fb2 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:05.265992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb6e19-5c12-45c4-8834-59b38b867fb2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:05.266292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb6e19-5c12-45c4-8834-59b38b867fb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:05.267891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb6e19-5c12-45c4-8834-59b38b867fb2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:05.273779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 364/723] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Oct 27 13:12:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:05.281301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:05.392217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e832f-507c-4bf3-b9fe-1e4b8003c152 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:05.431373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88fadd-d492-401e-ad84-dfc0b954d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:05.431979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88fadd-d492-401e-ad84-dfc0b954d47a None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:05.432259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88fadd-d492-401e-ad84-dfc0b954d47a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:05.432532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88fadd-d492-401e-ad84-dfc0b954d47a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:05.433180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88fadd-d492-401e-ad84-dfc0b954d47a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:05.434527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88fadd-d492-401e-ad84-dfc0b954d47a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:05.439047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:05.462328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 360/724] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:12:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:06.072281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aebcd4-eb39-46fb-8719-90a353ed6408 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.072807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aebcd4-eb39-46fb-8719-90a353ed6408 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.072911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aebcd4-eb39-46fb-8719-90a353ed6408 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.074477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 365/725] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:06.080291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ac1e8-b205-45e0-aa3d-7457cd8b37e3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.080608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ac1e8-b205-45e0-aa3d-7457cd8b37e3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.080845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ac1e8-b205-45e0-aa3d-7457cd8b37e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.100019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5a7ac1e8-b205-45e0-aa3d-7457cd8b37e3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:06.101790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5a7ac1e8-b205-45e0-aa3d-7457cd8b37e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:06.176700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 361/726] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:06.188429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125cd8a-01b1-4fdb-98bb-0ef90bcbae8d None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.188709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125cd8a-01b1-4fdb-98bb-0ef90bcbae8d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.188980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125cd8a-01b1-4fdb-98bb-0ef90bcbae8d None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.190429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 366/727] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:06.195751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283dcf1-809d-4eaa-934e-210b370c4937 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.196042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283dcf1-809d-4eaa-934e-210b370c4937 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.196341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283dcf1-809d-4eaa-934e-210b370c4937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.215453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2283dcf1-809d-4eaa-934e-210b370c4937 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:06.216706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2283dcf1-809d-4eaa-934e-210b370c4937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:06.285822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 362/728] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:06.455678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125cd8a-01b1-4fdb-98bb-0ef90bcbae8d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:06.560607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283dcf1-809d-4eaa-934e-210b370c4937 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:06.562457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125cd8a-01b1-4fdb-98bb-0ef90bcbae8d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:06.600147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:06.601837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.602335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.602802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] PATH_INFO: `/v3/services/image` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.603571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:06.605139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:06.607369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:06.615775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:06.617007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:06.617841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:06.618568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 367/729] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Oct 27 13:12:06 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:06.623823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:06.629125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:06.695838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283dcf1-809d-4eaa-934e-210b370c4937 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:06.731606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76a98e-a495-48a3-8d71-1a74c4c764e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:06.732250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76a98e-a495-48a3-8d71-1a74c4c764e9 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.732839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76a98e-a495-48a3-8d71-1a74c4c764e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.732949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76a98e-a495-48a3-8d71-1a74c4c764e9 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.733642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76a98e-a495-48a3-8d71-1a74c4c764e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:06.735073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76a98e-a495-48a3-8d71-1a74c4c764e9 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:06.760284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 363/730] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:12:06 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:06.760284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a0bc8-5a2e-4a0e-a985-1a126ff0be22 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.760284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a0bc8-5a2e-4a0e-a985-1a126ff0be22 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.760284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a0bc8-5a2e-4a0e-a985-1a126ff0be22 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.761556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e83d9-80ff-4f10-8441-e19c1f5f21fe None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:06.761664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 364/731] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:12:06.767855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950cc63-cea1-4543-b6d4-24e4b3ae02df None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.768170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950cc63-cea1-4543-b6d4-24e4b3ae02df None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.768432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950cc63-cea1-4543-b6d4-24e4b3ae02df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.795297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7950cc63-cea1-4543-b6d4-24e4b3ae02df None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:06.797063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7950cc63-cea1-4543-b6d4-24e4b3ae02df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:06.821434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:06.822069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.822345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.822614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.823253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] RBAC: Adding query filter params (name=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:06.823568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:06.825019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:06.832312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 368/732] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:06.838077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:06.905384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 365/733] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:06.917691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:06.963180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1c94f-1847-48c9-a79d-d718549fd206 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:06.996980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:06.997517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:06.997807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:06.998064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:06.998686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] RBAC: Adding query filter params (type=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:06.998985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:07.000365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:07.005279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 369/734] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:07.189839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950cc63-cea1-4543-b6d4-24e4b3ae02df None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:07.225899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:07.296203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950cc63-cea1-4543-b6d4-24e4b3ae02df None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:07.332178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:07.332891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:07.333179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:07.333454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:07.334041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:07.335538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:07.345894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:07.346896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:07.347896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 366/735] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:12:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:07.355292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:07.462087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffc392-83ea-4ae4-b085-0a4eac19c729 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:07.489133 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:07.498673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:07.499295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:07.499574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:07.499849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:07.500502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:07.500805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:07.502168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:07.512003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 370/736] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:07.515254 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:07.617389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:07.620894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6f6b8-8d9a-4cea-b232-5a3423c41b97 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:07.659565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc1dff-7430-47d1-8cc5-9cd5d6a26728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:07.660554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc1dff-7430-47d1-8cc5-9cd5d6a26728 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:07.661021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc1dff-7430-47d1-8cc5-9cd5d6a26728 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:07.661501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc1dff-7430-47d1-8cc5-9cd5d6a26728 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:07.662326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc1dff-7430-47d1-8cc5-9cd5d6a26728 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:07.662809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc1dff-7430-47d1-8cc5-9cd5d6a26728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:07.664965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc1dff-7430-47d1-8cc5-9cd5d6a26728 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:07.671176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 367/737] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:07.677139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:07.755466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e90dc-669a-4096-a134-9829966463b7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:07.755832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e90dc-669a-4096-a134-9829966463b7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:07.756029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e90dc-669a-4096-a134-9829966463b7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:07.757957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 368/738] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:07.763916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43927c-b296-494a-9c09-373921a2aacf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:07.764280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43927c-b296-494a-9c09-373921a2aacf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:07.764506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43927c-b296-494a-9c09-373921a2aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:07.783968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4c43927c-b296-494a-9c09-373921a2aacf None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:07.785207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4c43927c-b296-494a-9c09-373921a2aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:07.786627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d671a-afe9-41e5-bd4c-20bd8c85cc91 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:07.824570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:07.825179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:07.825456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:07.825726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:07.826426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] RBAC: Adding query filter params (interface=admin, service_id=951f3a243b804d408ad6c83c1da4d799, region_id=RegionOne) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:07.826737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:07.828314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:07.839160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 371/739] 10.222.0.48 () {56 vars in 1249 bytes} [Thu Oct 27 13:12:07 2022] GET /identity/v3/endpoints?service_id=951f3a243b804d408ad6c83c1da4d799&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:07.840558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:07.857443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 369/740] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:07.893762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:07.945214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d906b0-6172-4fbb-9d67-bdad6fc64320 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:07.982550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:07.983161 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:07.983439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:07.983712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:07.984340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:07.985697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:07.996545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 372/741] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:12:07 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:12:08.141426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43927c-b296-494a-9c09-373921a2aacf None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:08.238651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:08.247394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43927c-b296-494a-9c09-373921a2aacf None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:08.282546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:08.283200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:08.283478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:08.283741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:08.284406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:08.285813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:08.288931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING keystone.server.flask.common [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 27 13:12:08.308463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.sql.core [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 27 13:12:08.308463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 13:12:08.308463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [parameters: {'id': 'd9ccbdbceb364baebc0dbe8a7c536948', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 27 13:12:08.308463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9ccbdbceb364baebc0dbe8a7c536948.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9ccbdbceb364baebc0dbe8a7c536948. Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:12:08.325759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 27 13:12:08.326587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application yield resource Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise self.value Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application session.commit() Oct 27 13:12:08.327688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.flush() Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application compat.raise_( Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 13:12:08.328813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 13:12:08.329707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:12:08.330498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [parameters: {'id': 'd9ccbdbceb364baebc0dbe8a7c536948', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:08.331374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:08.333714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9ccbdbceb364baebc0dbe8a7c536948. Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:08.336012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 370/742] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Oct 27 13:12:08 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:08.344466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:08.440862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa79a8-d760-4c4a-906d-eaecbc4ee805 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:08.456626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:08.476320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a09d52-593f-4def-984a-8e2c7f5bffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:08.477023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a09d52-593f-4def-984a-8e2c7f5bffa7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:08.477707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a09d52-593f-4def-984a-8e2c7f5bffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:08.477985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a09d52-593f-4def-984a-8e2c7f5bffa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:08.485085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a09d52-593f-4def-984a-8e2c7f5bffa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:08.486525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a09d52-593f-4def-984a-8e2c7f5bffa7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-46a09d52-593f-4def-984a-8e2c7f5bffa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:12:08.498690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:08.499560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:08.500494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 373/743] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:12:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:08.507630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:08.617509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f9cf-3330-414b-8c52-caa938227655 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:08.652628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487b8ab-65bb-47ac-9262-68d5372eba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:08.653239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b8ab-65bb-47ac-9262-68d5372eba0a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:08.653513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b8ab-65bb-47ac-9262-68d5372eba0a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:08.653778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b8ab-65bb-47ac-9262-68d5372eba0a None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:08.654444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487b8ab-65bb-47ac-9262-68d5372eba0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:08.654740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487b8ab-65bb-47ac-9262-68d5372eba0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:08.656427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487b8ab-65bb-47ac-9262-68d5372eba0a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:08.677324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 371/744] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:12:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:09.094650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dce00-46f9-4cef-888a-f75a1922311b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.095033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dce00-46f9-4cef-888a-f75a1922311b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.095209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dce00-46f9-4cef-888a-f75a1922311b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.096791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 374/745] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:09.102306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8dce-d88b-4bc9-b2aa-e8e7f7f76dff None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.102592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8dce-d88b-4bc9-b2aa-e8e7f7f76dff None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.102866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8dce-d88b-4bc9-b2aa-e8e7f7f76dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.122044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b84d8dce-d88b-4bc9-b2aa-e8e7f7f76dff None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:09.123609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b84d8dce-d88b-4bc9-b2aa-e8e7f7f76dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:09.127054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:09.197979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 372/746] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:09.248809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2103e28-a676-466b-a702-b8b6f448fb19 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.249120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2103e28-a676-466b-a702-b8b6f448fb19 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.249392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2103e28-a676-466b-a702-b8b6f448fb19 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.250715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 375/747] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:09.255935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc793a-2b5f-4f47-bb71-6c63df282971 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.256217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc793a-2b5f-4f47-bb71-6c63df282971 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.256519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc793a-2b5f-4f47-bb71-6c63df282971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.276117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-34bc793a-2b5f-4f47-bb71-6c63df282971 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:09.277303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-34bc793a-2b5f-4f47-bb71-6c63df282971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:09.344046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 373/748] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:09.487104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2103e28-a676-466b-a702-b8b6f448fb19 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:09.587743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2103e28-a676-466b-a702-b8b6f448fb19 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:09.622200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:09.622838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.623111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.623379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.623948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:09.625274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc793a-2b5f-4f47-bb71-6c63df282971 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:09.625698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:09.635367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:09.636492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:09.637345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 376/749] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:12:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:09.643173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:09.692185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:09.701003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:09.733846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc793a-2b5f-4f47-bb71-6c63df282971 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:09.770821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:09.771353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.771641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.771901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.772639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:09.774011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:09.774521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796dacf9-f9a3-4ec3-ab9a-3ae5b836f125 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:09.796752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 374/750] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:12:09 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:09.820886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:09.821862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.822319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.822783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.823607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:09.824093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:09.825687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:09.832168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 377/751] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:12:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:09.838558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:09.955878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d952c-731a-4f5d-a837-52243388b0b3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:09.988693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:09.996081 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:09.996902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:09.997371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:09.997801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:09.998635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:09.999078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:10.000614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:10.008267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 375/752] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:12:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:10.011891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:10.119123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:10.123092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0995a-b2a5-4c5a-94f8-159e87d0d179 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:10.158523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d83e7-9d93-48ca-a569-768522e8ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:10.159129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d83e7-9d93-48ca-a569-768522e8ba96 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:10.159400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d83e7-9d93-48ca-a569-768522e8ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:10.159664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d83e7-9d93-48ca-a569-768522e8ba96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:10.160383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d83e7-9d93-48ca-a569-768522e8ba96 None admin] RBAC: Adding query filter params (interface=public, service_id=083ae82930d74931b62ef6b1d90dfd33, region_id=RegionOne) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:10.160687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d83e7-9d93-48ca-a569-768522e8ba96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:10.162027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d83e7-9d93-48ca-a569-768522e8ba96 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:10.168074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 378/753] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:12:10 2022] GET /identity/v3/endpoints?service_id=083ae82930d74931b62ef6b1d90dfd33&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:10.173569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:10.285119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b4017-3ad9-4ba9-8d93-b871499ca98e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:10.319872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a70f55-8052-4f65-8ba3-71b15276785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:10.320503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a70f55-8052-4f65-8ba3-71b15276785d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:10.320774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a70f55-8052-4f65-8ba3-71b15276785d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:10.321039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a70f55-8052-4f65-8ba3-71b15276785d None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:10.321686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a70f55-8052-4f65-8ba3-71b15276785d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:10.323038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a70f55-8052-4f65-8ba3-71b15276785d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:10.329965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 376/754] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:12:10 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:12:10.337039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcb5cf-0074-4988-b487-4047a1102e43 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:10.337323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcb5cf-0074-4988-b487-4047a1102e43 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:10.337595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcb5cf-0074-4988-b487-4047a1102e43 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:10.340554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 379/755] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:10.345349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f40b0-a78e-4c27-981c-3a561cad3f71 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:10.345641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f40b0-a78e-4c27-981c-3a561cad3f71 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:10.345911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f40b0-a78e-4c27-981c-3a561cad3f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:10.367006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0a1f40b0-a78e-4c27-981c-3a561cad3f71 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:10.368635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0a1f40b0-a78e-4c27-981c-3a561cad3f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:10.429981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:10.447347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 377/756] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:10 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:10.713903 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcb5cf-0074-4988-b487-4047a1102e43 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:10.827646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcb5cf-0074-4988-b487-4047a1102e43 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:10.862576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ac27b-b52b-4534-b95b-63e580d3f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:10.863204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac27b-b52b-4534-b95b-63e580d3f853 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:10.863476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac27b-b52b-4534-b95b-63e580d3f853 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:10.863742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ac27b-b52b-4534-b95b-63e580d3f853 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:10.864345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ac27b-b52b-4534-b95b-63e580d3f853 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:10.865834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ac27b-b52b-4534-b95b-63e580d3f853 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-cb0ac27b-b52b-4534-b95b-63e580d3f853 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:10.875596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:10.876699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:10.877802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 380/757] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Oct 27 13:12:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:10.886029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f40b0-a78e-4c27-981c-3a561cad3f71 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:10.993169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f40b0-a78e-4c27-981c-3a561cad3f71 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:11.002960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f3627-84da-4f89-a262-45ea4547d4ef None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.003246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f3627-84da-4f89-a262-45ea4547d4ef None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.003762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f3627-84da-4f89-a262-45ea4547d4ef None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.005548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 381/758] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:11.011407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c956b-802c-4d63-9d2f-40ee150c9757 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.011940 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c956b-802c-4d63-9d2f-40ee150c9757 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.012317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c956b-802c-4d63-9d2f-40ee150c9757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.029439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:11.030113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.030395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.030663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.031338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:11.031640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:11.033028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:11.034071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e29c956b-802c-4d63-9d2f-40ee150c9757 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:11.035855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e29c956b-802c-4d63-9d2f-40ee150c9757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:11.042964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 378/759] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:11.044179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:11.059163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:11.104559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:11.121503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 382/760] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:11 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:11.156572 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abc903-fce3-4876-8e5c-80c557048963 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:11.190473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:11.191074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.191346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.191608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.192242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:11.192543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:11.193901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:11.199732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 379/761] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Oct 27 13:12:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:11.205302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c956b-802c-4d63-9d2f-40ee150c9757 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:11.314104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c956b-802c-4d63-9d2f-40ee150c9757 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:11.347882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:11.348786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.349296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.349783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.351168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:11.352817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:11.374551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 383/762] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:12:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:12:11.420555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:11.537527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d62c6a-c3a0-46fe-bfcc-ab82cc48f6ab None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:11.560706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:11.583989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:11.584598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.584877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.585694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.586362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:11.587799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:11.607919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.sql.core [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 27 13:12:11.607919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 27 13:12:11.607919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [parameters: {'user_id': 'a75dda3f22d04b439fb4226f9dea8e99', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 27 13:12:11.607919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Oct 27 13:12:11.616826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:12:11.617881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application yield resource Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise self.value Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application session.commit() Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 13:12:11.619506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.flush() Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application compat.raise_( Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 13:12:11.620725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:12:11.621900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:12:11.622965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a75dda3f22d04b439fb4226f9dea8e99', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:11.623982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 27 13:12:11.629239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 27 13:12:11.630467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 27 13:12:11.630467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 27 13:12:11.630467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 13:12:11.630467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 13:12:11.630467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 27 13:12:11.630467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:11.630467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 380/763] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Oct 27 13:12:11 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:11.633151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:11.658036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:11.744011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6549ee9-0f12-4ee5-8b8e-5a19a033d268 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:11.782897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a048575-0b4e-43e8-a6b2-c840dc2e38bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:11.783528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a048575-0b4e-43e8-a6b2-c840dc2e38bc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.783800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a048575-0b4e-43e8-a6b2-c840dc2e38bc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.784068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a048575-0b4e-43e8-a6b2-c840dc2e38bc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.792147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a048575-0b4e-43e8-a6b2-c840dc2e38bc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:11.793597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a048575-0b4e-43e8-a6b2-c840dc2e38bc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-2a048575-0b4e-43e8-a6b2-c840dc2e38bc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:12:11.805952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:11.807835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:11.809915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 384/764] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:12:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:11.820260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:11.921896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114fe3c-23e1-4977-9414-274c35cf75e3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:11.963625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e8a6-6c79-4688-a5b9-3a472b0dd241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:11.964257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e8a6-6c79-4688-a5b9-3a472b0dd241 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:11.964594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e8a6-6c79-4688-a5b9-3a472b0dd241 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:11.964870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e8a6-6c79-4688-a5b9-3a472b0dd241 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:11.965568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37e8a6-6c79-4688-a5b9-3a472b0dd241 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:11.965868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37e8a6-6c79-4688-a5b9-3a472b0dd241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:11.967488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37e8a6-6c79-4688-a5b9-3a472b0dd241 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:12.006741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 381/765] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Oct 27 13:12:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:12.244258 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf391cc9-0770-400c-9c1b-a907a9b7a38e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:12.244719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf391cc9-0770-400c-9c1b-a907a9b7a38e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:12.245091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf391cc9-0770-400c-9c1b-a907a9b7a38e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:12.247217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 385/766] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:12.253256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abe711-7a12-4435-b270-3ca3918974a8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:12.253545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abe711-7a12-4435-b270-3ca3918974a8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:12.253824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abe711-7a12-4435-b270-3ca3918974a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:12.273691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-88abe711-7a12-4435-b270-3ca3918974a8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:12.275197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-88abe711-7a12-4435-b270-3ca3918974a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:12.302915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:12.352197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 382/767] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:12.579237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f61c67-0b11-4c36-8104-0cc6e5de3f88 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:12.579424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f61c67-0b11-4c36-8104-0cc6e5de3f88 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:12.579718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f61c67-0b11-4c36-8104-0cc6e5de3f88 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:12.581182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 386/768] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:12.586629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf94dc-0912-41e9-98ad-4c86aa9ee8b5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:12.586912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf94dc-0912-41e9-98ad-4c86aa9ee8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:12.587181 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf94dc-0912-41e9-98ad-4c86aa9ee8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:12.605800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-18bf94dc-0912-41e9-98ad-4c86aa9ee8b5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:12.606961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-18bf94dc-0912-41e9-98ad-4c86aa9ee8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:12.621548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f61c67-0b11-4c36-8104-0cc6e5de3f88 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:12.678684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 383/769] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:12.702984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:12.728143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f61c67-0b11-4c36-8104-0cc6e5de3f88 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:12.763229 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:12.763861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:12.764115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:12.764411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] PATH_INFO: `/v3/services/image` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:12.764983 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:12.766326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:12.775779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:12.776930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:12.778213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:12.780570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 387/770] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Oct 27 13:12:12 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:12.782771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf94dc-0912-41e9-98ad-4c86aa9ee8b5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:12.871718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:12.881392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf94dc-0912-41e9-98ad-4c86aa9ee8b5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:12.915427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:12.916029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:12.916338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:12.916610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:12.917231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] RBAC: Adding query filter params (name=image) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:12.917530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:12.918888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:12.923922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 384/771] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:12.929444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:12.950795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:13.031031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422ad48-4e70-40b5-9a79-a9f83bb5c292 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:13.050337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88288f2-f7be-4b50-af7f-08d39ebd3469 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:13.064906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:13.065586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.065877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.066130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.066740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:13.067035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:13.068399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:13.074086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 388/772] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:13.079689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:13.083375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:13.094074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:13.094820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.095104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.095376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.095955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:13.097360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:13.106153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:13.107165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:13.108141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 385/773] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:12:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:13.113145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:13.136425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:13.188404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf91685-8397-4dc6-bd69-a858696818f6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:13.218449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1354b4e-901a-4773-8892-744af67d56cb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:13.223557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:13.224298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.224676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.225047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.225957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] RBAC: Adding query filter params (interface=public, service_id=98057820bc824dd980c82fb66155f06d, region_id=RegionOne) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:13.226356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:13.227833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:13.237979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 389/774] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:12:13 2022] GET /identity/v3/endpoints?service_id=98057820bc824dd980c82fb66155f06d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:13.238925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:13.253042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:13.253733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.254073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.254343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.254983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:13.255283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:13.256663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:13.262127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 386/775] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:12:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:13.267807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:13.313662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:13.348107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc96c8c-f3d7-48c7-aedb-0e8ec47b6374 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:13.351312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:13.368133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eed4e7-f684-44b7-bc8d-970b632df118 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:13.385833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:13.386490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.386705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.386969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.387552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:13.390388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:13.399841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 390/776] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:12:13 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:12:13.408856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49feaf-0120-40df-983e-6bdfef2f7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:13.409490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49feaf-0120-40df-983e-6bdfef2f7589 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.409774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49feaf-0120-40df-983e-6bdfef2f7589 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.410045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49feaf-0120-40df-983e-6bdfef2f7589 None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.410693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49feaf-0120-40df-983e-6bdfef2f7589 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:13.410997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49feaf-0120-40df-983e-6bdfef2f7589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:13.412424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49feaf-0120-40df-983e-6bdfef2f7589 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:13.422862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 387/777] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Oct 27 13:12:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:13.425403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:13.541027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff94422-13f6-4d9b-8dce-9251b2a005af None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:13.571920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:13.581075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115afdc-0aaf-4cf2-b684-f131fe52248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:13.581690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115afdc-0aaf-4cf2-b684-f131fe52248a None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.581965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115afdc-0aaf-4cf2-b684-f131fe52248a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.582230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115afdc-0aaf-4cf2-b684-f131fe52248a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.582894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115afdc-0aaf-4cf2-b684-f131fe52248a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:13.584254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115afdc-0aaf-4cf2-b684-f131fe52248a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:13.606198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 391/778] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:12:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:13.624773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c72d-4d40-459b-947b-a1f8e8300808 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.625083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c72d-4d40-459b-947b-a1f8e8300808 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.625336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c72d-4d40-459b-947b-a1f8e8300808 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.626708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 388/779] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:13.632873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c6d82-45c4-4cc6-aecf-21ffcece9ccf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:13.633228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c6d82-45c4-4cc6-aecf-21ffcece9ccf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:13.633445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c6d82-45c4-4cc6-aecf-21ffcece9ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:13.652978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-328c6d82-45c4-4cc6-aecf-21ffcece9ccf None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:13.654087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-328c6d82-45c4-4cc6-aecf-21ffcece9ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:13.740257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 392/780] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:14.001191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c72d-4d40-459b-947b-a1f8e8300808 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:14.101904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c72d-4d40-459b-947b-a1f8e8300808 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:14.135119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea8f1d-1168-4604-9cae-b65f20b2f1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:14.135752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea8f1d-1168-4604-9cae-b65f20b2f1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:14.136024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea8f1d-1168-4604-9cae-b65f20b2f1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:14.136311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea8f1d-1168-4604-9cae-b65f20b2f1b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:14.137407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea8f1d-1168-4604-9cae-b65f20b2f1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:14.138810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea8f1d-1168-4604-9cae-b65f20b2f1b2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:14.140603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 389/781] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:14.148400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c6d82-45c4-4cc6-aecf-21ffcece9ccf None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:14.250352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c6d82-45c4-4cc6-aecf-21ffcece9ccf None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:14.266104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:14.288567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26e973-18b7-4aef-9869-db38a15d6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:14.289316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26e973-18b7-4aef-9869-db38a15d6e99 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:14.289696 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26e973-18b7-4aef-9869-db38a15d6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:14.290067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26e973-18b7-4aef-9869-db38a15d6e99 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:14.290805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26e973-18b7-4aef-9869-db38a15d6e99 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:14.292338 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26e973-18b7-4aef-9869-db38a15d6e99 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:14.328305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 393/782] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:12:14 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:15.597334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84169e96-4cb1-4ae4-897c-bf080dc3eb03 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:15.597879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84169e96-4cb1-4ae4-897c-bf080dc3eb03 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:15.597879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84169e96-4cb1-4ae4-897c-bf080dc3eb03 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:15.599231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 390/783] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:15.605004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f16b9a-27e1-4f14-bbd5-fdc76868b0b1 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:15.605384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f16b9a-27e1-4f14-bbd5-fdc76868b0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:15.605567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f16b9a-27e1-4f14-bbd5-fdc76868b0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:15.626470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-05f16b9a-27e1-4f14-bbd5-fdc76868b0b1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:15.627631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-05f16b9a-27e1-4f14-bbd5-fdc76868b0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:15.700286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 394/784] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:15.820006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d65bf0-9ba1-4fe9-ba77-61a690ef0016 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:15.820364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d65bf0-9ba1-4fe9-ba77-61a690ef0016 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:15.820588 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d65bf0-9ba1-4fe9-ba77-61a690ef0016 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:15.822119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 391/785] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:15.826995 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d000ae9-afcd-4edc-95b2-ff65146f2677 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:15.827346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d000ae9-afcd-4edc-95b2-ff65146f2677 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:15.827616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d000ae9-afcd-4edc-95b2-ff65146f2677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:15.845792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7d000ae9-afcd-4edc-95b2-ff65146f2677 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:15.846960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7d000ae9-afcd-4edc-95b2-ff65146f2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:15.912795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 395/786] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:15.963689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d65bf0-9ba1-4fe9-ba77-61a690ef0016 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:15.982017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:16.066020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d65bf0-9ba1-4fe9-ba77-61a690ef0016 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.100113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.100789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.101063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.101330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] PATH_INFO: `/v3/services/image` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.101907 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:16.103265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:16.112023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:16.113202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 27 13:12:16.114025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:16.116910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 392/787] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Oct 27 13:12:15 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:16.119454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d000ae9-afcd-4edc-95b2-ff65146f2677 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:16.147030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:16.176446 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:16.224577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d000ae9-afcd-4edc-95b2-ff65146f2677 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.261238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.261838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.262276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.262471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:16.262574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.263147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] RBAC: Adding query filter params (name=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:16.263827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:16.265381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:16.270616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 396/788] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:16.276353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:16.283913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4195ab-4fb8-48ee-b832-3683d1007e50 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.320053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.321018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.321357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.321699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.331658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:16.336535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:16.337442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:16.338301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 393/789] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Oct 27 13:12:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:16.343483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:16.375912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:16.387437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005b752-1dc6-48c8-bc59-bacc8339f68c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.425939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.426760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.427020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.427299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.427937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] RBAC: Adding query filter params (type=image) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:16.428256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:16.430191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:16.440757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 397/790] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:16.441866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:16.454794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90d844-d9e2-418e-9810-2668461e5b7e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.489674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.490511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.490790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.491666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.492319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:16.492627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:16.493991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:16.506513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 394/791] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Oct 27 13:12:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:16.510032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:16.513433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:16.546253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4052dcf-86b7-4263-a710-435f409ae495 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.578536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3682f4-d082-4a63-9b00-09f6082a2a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.579130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3682f4-d082-4a63-9b00-09f6082a2a71 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.579401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3682f4-d082-4a63-9b00-09f6082a2a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.579664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3682f4-d082-4a63-9b00-09f6082a2a71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.580345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3682f4-d082-4a63-9b00-09f6082a2a71 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:16.581679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3682f4-d082-4a63-9b00-09f6082a2a71 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:16.602559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 398/792] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:12:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:16.603848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038b822-03b1-4f74-850c-67738a1b2ccc None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.604127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038b822-03b1-4f74-850c-67738a1b2ccc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.604420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038b822-03b1-4f74-850c-67738a1b2ccc None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.605770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 399/793] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:16.611538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781a79-60ba-4804-a69d-a25ce97ee35f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.611868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781a79-60ba-4804-a69d-a25ce97ee35f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.612096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781a79-60ba-4804-a69d-a25ce97ee35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.614337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f721521-98c7-42e6-9f43-8ad77a554c8c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.639196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-90781a79-60ba-4804-a69d-a25ce97ee35f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:16.641882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-90781a79-60ba-4804-a69d-a25ce97ee35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:16.646853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:16.661004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.661655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.661931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.662214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.675126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:16.677204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:12:16.693390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:16.694482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:12:16.695361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:12:16.695361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:16.695361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:16.696216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 395/794] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:16.702959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:16.727529 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 400/795] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:16.758327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:16.813335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27160c-e6ab-4b6c-b556-674918ee3d1e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:16.851536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:16.852141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:16.852443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:16.852712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:16.853350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:16.853646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:16.855009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:16.867050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 396/796] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:16.872452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781a79-60ba-4804-a69d-a25ce97ee35f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.001934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.092695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781a79-60ba-4804-a69d-a25ce97ee35f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:17.108338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777ea33-1273-4052-b176-96cc9b8f5688 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:17.119514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:17.133042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:17.134123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:17.134400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:17.134935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:17.143011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:17.144448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:17.145256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:17.145889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:17.146172 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:17.146445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:17.148058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:17.148470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:17.149538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:17.151239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 397/797] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:17.156937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:12:17.159093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:17.160179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:12:17.161324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:17.165850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 401/798] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Oct 27 13:12:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:17.169687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.269373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfa84-0a5c-4d6e-90b2-692fbc0c4344 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:17.276739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14705c12-b838-4f29-9922-d4de8b6f4264 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:17.310614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:17.311246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:17.311519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:17.311785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:17.312440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:17.312556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:17.312826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:17.313087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:17.313364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:17.313629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:17.314235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:17.314332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:17.315662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:17.333920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 402/799] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Oct 27 13:12:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:17.340144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.348440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 398/800] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:12:17 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:12:17.412703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:17.459713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca87b9-f05e-4253-a8d2-a235d05cd7c0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:17.497297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:17.497941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:17.498213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:17.498475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:17.510215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:17.511665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:12:17.528328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:17.529490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:12:17.530602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:12:17.530602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:17.530602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:17.532510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 403/801] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:17.538240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.575671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:17.645687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720c753-02ad-46fb-9dd5-e023d13582e7 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:17.679921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:17.680550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:17.680825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:17.681089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:17.681722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:17.682018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:17.683358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:17.700105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 399/802] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:17.701481 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.801978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c652463-f23a-457b-b58b-70e072d70af6 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:17.812036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:17.836520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:17.837147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:17.837418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:17.837681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:17.844289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:17.845717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:12:17.857414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:17.858306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:17.859210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:17.860362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 404/803] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:17.864256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:17.962132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:17.970787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a2ba6-f10b-4ea6-a966-078d0d632fa3 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:18.004065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37466edb-1485-4a5c-a927-4eb13d40c6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:18.004675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37466edb-1485-4a5c-a927-4eb13d40c6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:18.004954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37466edb-1485-4a5c-a927-4eb13d40c6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:18.005223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37466edb-1485-4a5c-a927-4eb13d40c6f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:18.005904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37466edb-1485-4a5c-a927-4eb13d40c6f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:18.006203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37466edb-1485-4a5c-a927-4eb13d40c6f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:18.008210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37466edb-1485-4a5c-a927-4eb13d40c6f0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:18.025354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 400/804] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:12:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:18.026574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:18.125204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c117c-e024-4dec-ab56-282696f726d5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:18.158935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af7e85-1805-44e5-872b-de170cf7f206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:18.159582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7e85-1805-44e5-872b-de170cf7f206 None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:18.159856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7e85-1805-44e5-872b-de170cf7f206 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:18.160123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7e85-1805-44e5-872b-de170cf7f206 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/d1d142bc3a9e4b3badefe713c7405329/roles/60e2466e171e4daca6ebf84d6504372f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:18.162247 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af7e85-1805-44e5-872b-de170cf7f206 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=d1d142bc3a9e4b3badefe713c7405329, role_id=60e2466e171e4daca6ebf84d6504372f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:18.163692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af7e85-1805-44e5-872b-de170cf7f206 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:18.176188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 405/805] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:12:18 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/d1d142bc3a9e4b3badefe713c7405329/roles/60e2466e171e4daca6ebf84d6504372f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:12:18.808706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bdcf0-4f3d-463d-8050-ae104205dcec None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:18.809214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bdcf0-4f3d-463d-8050-ae104205dcec None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:18.809214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bdcf0-4f3d-463d-8050-ae104205dcec None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:18.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 401/806] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:18.815721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18867f1-8ecb-4d66-b8f2-afea04f1c1e6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:18.816093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18867f1-8ecb-4d66-b8f2-afea04f1c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:18.816273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18867f1-8ecb-4d66-b8f2-afea04f1c1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:18.833903 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b18867f1-8ecb-4d66-b8f2-afea04f1c1e6 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:18.835070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b18867f1-8ecb-4d66-b8f2-afea04f1c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:18.858364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:18.904396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 406/807] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:19.173054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bdcf0-4f3d-463d-8050-ae104205dcec None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:19.274360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bdcf0-4f3d-463d-8050-ae104205dcec None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:19.307212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:19.307810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:19.308082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:19.308372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:19.309036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:19.310375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:19.319049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.sql.core [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 13:12:19.319049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 13:12:19.319049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [parameters: {'id': 'cb58b54b95e54f94a80f7ba8f5ef4446', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 13:12:19.319049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:12:19.326263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 27 13:12:19.327080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application yield resource Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application next(self.gen) Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.rollback() Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.force_reraise() Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise self.value Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application session.commit() Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 27 13:12:19.328707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._prepare_impl() Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.session.flush() Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._flush(objects) Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application compat.raise_( Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application flush_context.execute() Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rec.execute(self) Oct 27 13:12:19.329579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 27 13:12:19.330443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result = self._query(query) Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application conn.query(q) Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application result.read() Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 27 13:12:19.331205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [parameters: {'id': 'cb58b54b95e54f94a80f7ba8f5ef4446', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 27 13:12:19.332118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:19.332963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:19.335205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 402/808] 10.222.0.48 () {60 vars in 1137 bytes} [Thu Oct 27 13:12:19 2022] POST /identity/v3/domains => generated 190 bytes in 155 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:19.336746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18867f1-8ecb-4d66-b8f2-afea04f1c1e6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:19.438651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18867f1-8ecb-4d66-b8f2-afea04f1c1e6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:19.476479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc242697-3585-4177-a039-adfe7b9b39d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:19.477128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242697-3585-4177-a039-adfe7b9b39d4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:19.477402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242697-3585-4177-a039-adfe7b9b39d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:19.477683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242697-3585-4177-a039-adfe7b9b39d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:19.488587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc242697-3585-4177-a039-adfe7b9b39d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:19.490020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc242697-3585-4177-a039-adfe7b9b39d4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-dc242697-3585-4177-a039-adfe7b9b39d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:12:19.504131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:19.505104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:12:19.505966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:12:19.505966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:19.505966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:19.508299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 407/809] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:19.512052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:19.533001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:19.560763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f769a-3c32-4ef1-86d7-87e5c0b6c95f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:19.561043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f769a-3c32-4ef1-86d7-87e5c0b6c95f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:19.561310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f769a-3c32-4ef1-86d7-87e5c0b6c95f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:19.563434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 408/810] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:12:19.568281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61597f-c412-4cdb-bf41-ec0bb17c1fea None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:19.568566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61597f-c412-4cdb-bf41-ec0bb17c1fea None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:19.568834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61597f-c412-4cdb-bf41-ec0bb17c1fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:19.588293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bb61597f-c412-4cdb-bf41-ec0bb17c1fea None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:19.589468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bb61597f-c412-4cdb-bf41-ec0bb17c1fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:19.597456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:19.620438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abb3b9-1b4f-4bb2-a5ba-354c72170915 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:19.658236 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:19.658893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:19.659175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:19.659445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:19.660078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:19.660417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:19.661787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:19.663665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 409/811] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:19.684259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 403/812] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:19.937881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61597f-c412-4cdb-bf41-ec0bb17c1fea None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:20.036866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61597f-c412-4cdb-bf41-ec0bb17c1fea None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:20.069871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:20.070482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.070749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.071007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.080610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:20.085276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:20.086407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:20.087581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:20.088640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 410/813] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:20.092320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:20.168447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:20.191057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d8194-6453-4db1-86dc-93bd0860ba8e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:20.227772 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db29db-cecd-4182-ac15-8a42fd04ae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:20.228412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db29db-cecd-4182-ac15-8a42fd04ae4c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.228690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db29db-cecd-4182-ac15-8a42fd04ae4c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.228955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db29db-cecd-4182-ac15-8a42fd04ae4c None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.229606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db29db-cecd-4182-ac15-8a42fd04ae4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:20.229902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db29db-cecd-4182-ac15-8a42fd04ae4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:20.231253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db29db-cecd-4182-ac15-8a42fd04ae4c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:20.239352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 404/814] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:12:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:20.246205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:20.249647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:20.305861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3549b-9a6a-4d82-8954-12bac1b1c8df None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.306140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3549b-9a6a-4d82-8954-12bac1b1c8df None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.306409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3549b-9a6a-4d82-8954-12bac1b1c8df None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.307726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 405/815] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:12:20.312431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea2e09-5e3f-4133-b4bc-530f892301de None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.312706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea2e09-5e3f-4133-b4bc-530f892301de None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.312974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea2e09-5e3f-4133-b4bc-530f892301de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.330203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-73ea2e09-5e3f-4133-b4bc-530f892301de None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:20.331361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-73ea2e09-5e3f-4133-b4bc-530f892301de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:20.344046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a3e6f-ff0d-4190-b0e0-fcb083a58362 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:20.376689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:20.377297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.377566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.377833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] PATH_INFO: `/v3/users/163706e1ae664c4b867e11002f0ecd87` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.390287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] RBAC: Authorizing `identity:get_user(user_id=163706e1ae664c4b867e11002f0ecd87)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:20.391955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:20.393527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 411/816] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:20 2022] GET /identity/v3/users/163706e1ae664c4b867e11002f0ecd87 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:20.397232 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 406/817] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:20.401898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:20.457317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:20.496726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4c417-0b95-4dc3-90eb-559372c27311 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:20.528398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:20.529008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.529287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.529555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] PATH_INFO: `/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.530632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6a7621bc2447c78cf9267f8c5c2a2d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:20.532023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:20.533629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 412/818] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:12:20 2022] GET /identity/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:20.540061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea2e09-5e3f-4133-b4bc-530f892301de None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:20.607818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:20.638132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea2e09-5e3f-4133-b4bc-530f892301de None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:20.662643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:20.672128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:20.672815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.673093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.673362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] PATH_INFO: `/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d/users/163706e1ae664c4b867e11002f0ecd87/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.675265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6a7621bc2447c78cf9267f8c5c2a2d, user_id=163706e1ae664c4b867e11002f0ecd87, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:20.676733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:20.689247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 407/819] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:12:20 2022] PUT /identity/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d/users/163706e1ae664c4b867e11002f0ecd87/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:12:20.766497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6be8d-43fb-4078-a812-7f1fd9bd40d4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:20.794884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:20.803300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:20.803969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.804265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.804539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.816412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:20.821331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:20.822310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:20.823151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 413/820] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Oct 27 13:12:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:12:20.828838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:20.929970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7343a-68bf-4d5a-a5e5-89e12e02a112 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:20.957655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:20.964921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:20.965531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:20.965809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:20.966080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:20.966741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:20.967043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:20.968431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:20.976302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 408/821] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Oct 27 13:12:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:20.982169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:21.078727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294657-f346-40e4-a4f3-1625b1f37307 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:21.110855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:21.111472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:21.111748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:21.112018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:21.122186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:21.123608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:12:21.137794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:21.138915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:12:21.139848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:12:21.139848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:21.139848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:21.140415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 414/822] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:21.146582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:21.243796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c68f4-b853-4766-868e-789e0a53e462 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:21.277700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:21.278304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:21.278581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:21.278851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:21.279482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:21.279780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:21.281163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:21.292452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 409/823] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:21.298176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:21.338866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:21.394177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b694-a9c6-4a7d-847b-56a883930db4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:21.425669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:21.426280 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:21.426547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:21.426853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:21.434872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:21.436294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:12:21.447504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:21.448468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:21.449419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 415/824] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Oct 27 13:12:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:21.454388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:21.504626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:21.551509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37915409-47b2-4d12-967b-fc7fc2751050 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:21.589105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50020179-e600-4444-b683-31814eac9bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:21.589630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50020179-e600-4444-b683-31814eac9bca None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:21.589906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50020179-e600-4444-b683-31814eac9bca None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:21.590162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50020179-e600-4444-b683-31814eac9bca None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:21.590877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50020179-e600-4444-b683-31814eac9bca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:21.591174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50020179-e600-4444-b683-31814eac9bca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:21.592603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50020179-e600-4444-b683-31814eac9bca None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:21.609643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 410/825] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Oct 27 13:12:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:21.615226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:21.707884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:21.710805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fa243-2e08-4bdc-b7d5-3f1c47fed4e5 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:21.743495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:21.744081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:21.744385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:21.744648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:21.808458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:21.809867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:21.899629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:21.903850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 416/826] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:12:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:12:21.909576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50020179-e600-4444-b683-31814eac9bca None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:22.003569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50020179-e600-4444-b683-31814eac9bca None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:22.013657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:22.036656 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:22.037275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:22.037551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:22.038352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:22.047856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:22.049306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:12:22.063688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:22.069819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:12:22.070944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:12:22.070944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:12:22.070944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:22.070944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 411/827] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:22.071214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:22.166623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c1e72-e02b-47cf-84b3-467c076ae7a0 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:22.198548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:22.199136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:22.199404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:22.199663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:22.200298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:22.200591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:22.201913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:22.213329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 417/828] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:22.218649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:22.314490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20774735-c29b-45d6-a58e-1879e2430982 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:22.346420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:22.347036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:22.347307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:22.347572 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:22.354589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:22.355994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:12:22.366750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:22.367602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:22.368712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 412/829] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:22.373542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:22.433462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:22.474480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d8fcf-bfcc-4967-b543-e5a64ef84ead None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:22.506871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dca589-5116-4da8-a738-f4226f87959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:22.507482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dca589-5116-4da8-a738-f4226f87959e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:22.507755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dca589-5116-4da8-a738-f4226f87959e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:22.508018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dca589-5116-4da8-a738-f4226f87959e None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:22.508721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dca589-5116-4da8-a738-f4226f87959e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:22.509017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dca589-5116-4da8-a738-f4226f87959e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:22.510418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dca589-5116-4da8-a738-f4226f87959e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:22.527510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 418/830] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:22.528624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:22.552751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:22.625608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4120ca-005d-4316-a723-9370a40cbe1c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:22.658655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1840f-2258-464b-8149-d35c26ccc4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:22.659253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1840f-2258-464b-8149-d35c26ccc4de None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:22.659528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1840f-2258-464b-8149-d35c26ccc4de None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:22.659797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1840f-2258-464b-8149-d35c26ccc4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:22.660544 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1840f-2258-464b-8149-d35c26ccc4de None admin] RBAC: Adding query filter params (role.id=60e2466e171e4daca6ebf84d6504372f, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=d1d142bc3a9e4b3badefe713c7405329) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:22.660846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1840f-2258-464b-8149-d35c26ccc4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:22.662262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1840f-2258-464b-8149-d35c26ccc4de None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:22.667863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 413/831] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:12:22 2022] GET /identity/v3/role_assignments?user.id=d1d142bc3a9e4b3badefe713c7405329&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=60e2466e171e4daca6ebf84d6504372f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:22.833981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3c43a-d1a7-408b-88f4-501bbcaa656a None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:22.834178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3c43a-d1a7-408b-88f4-501bbcaa656a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:22.834428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3c43a-d1a7-408b-88f4-501bbcaa656a None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:22.836323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 419/832] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:22.841674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a9be5-a52a-4016-8ad2-204c4fc7ba65 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:22.841958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a9be5-a52a-4016-8ad2-204c4fc7ba65 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:22.842232 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a9be5-a52a-4016-8ad2-204c4fc7ba65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:22.861075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3d1a9be5-a52a-4016-8ad2-204c4fc7ba65 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:22.862252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3d1a9be5-a52a-4016-8ad2-204c4fc7ba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:22.929392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 414/833] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:23.195893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3c43a-d1a7-408b-88f4-501bbcaa656a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:23.298897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3c43a-d1a7-408b-88f4-501bbcaa656a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:23.319406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:23.333672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:23.334295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:23.334600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:23.334874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:23.344978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:23.349699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:23.351249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:23.352068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 420/834] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:12:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:23.357420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a9be5-a52a-4016-8ad2-204c4fc7ba65 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:23.376981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:23.466280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a9be5-a52a-4016-8ad2-204c4fc7ba65 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:23.614360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:23.614947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:23.615215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:23.615476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:23.616668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:23.617400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:23.618287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:23.625988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 415/835] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:12:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:23.632618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:23.726709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf6ae-5fee-4a63-b5c6-927d95ea85e1 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:23.759229 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:23.759826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:23.760125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:23.760421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] PATH_INFO: `/v3/users/163706e1ae664c4b867e11002f0ecd87` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:23.761507 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] RBAC: Authorizing `identity:get_user(user_id=163706e1ae664c4b867e11002f0ecd87)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:23.762900 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:23.764440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 421/836] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:23 2022] GET /identity/v3/users/163706e1ae664c4b867e11002f0ecd87 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:23.770222 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:23.843910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:23.864320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d128a-8b1e-4452-90cc-8d00295e62da None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:23.896514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:23.897123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:23.897391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:23.897650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:23.961169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:23.962593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:24.034526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:24.056585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 416/837] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:12:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:12:24.062396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:24.108493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:24.156591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8749b6e-ef71-46f0-91fd-f00134d125e9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:24.188491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d9de0-385d-471c-9fb3-8ae7d8471850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:24.189096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9de0-385d-471c-9fb3-8ae7d8471850 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:24.189369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9de0-385d-471c-9fb3-8ae7d8471850 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:24.189634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9de0-385d-471c-9fb3-8ae7d8471850 None admin] PATH_INFO: `/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:24.190709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d9de0-385d-471c-9fb3-8ae7d8471850 None admin] RBAC: Authorizing `identity:get_project(project_id=4b6a7621bc2447c78cf9267f8c5c2a2d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:24.192106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d9de0-385d-471c-9fb3-8ae7d8471850 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:24.193699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 422/838] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:12:24 2022] GET /identity/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:24.200279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:24.295096 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61876924-f1b6-4c44-9a39-8e47e121e6aa None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:24.328001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c550743-389a-44f6-bd62-db8c931aa31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:24.328619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550743-389a-44f6-bd62-db8c931aa31c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:24.328894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550743-389a-44f6-bd62-db8c931aa31c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:24.329157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550743-389a-44f6-bd62-db8c931aa31c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:24.329884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c550743-389a-44f6-bd62-db8c931aa31c None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=4b6a7621bc2447c78cf9267f8c5c2a2d, user.id=163706e1ae664c4b867e11002f0ecd87) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:24.330184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c550743-389a-44f6-bd62-db8c931aa31c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:24.331574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c550743-389a-44f6-bd62-db8c931aa31c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:24.337677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 417/839] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:12:24 2022] GET /identity/v3/role_assignments?user.id=163706e1ae664c4b867e11002f0ecd87&scope.project.id=4b6a7621bc2447c78cf9267f8c5c2a2d&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:26.460425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40ced-a54d-46b7-8f60-f39d4ea10877 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:26.460973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40ced-a54d-46b7-8f60-f39d4ea10877 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:26.460973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40ced-a54d-46b7-8f60-f39d4ea10877 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:26.462272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 423/840] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:26.467380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad83957-d8bd-47e1-8451-ef13a9a2d339 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:26.467754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad83957-d8bd-47e1-8451-ef13a9a2d339 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:26.467921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad83957-d8bd-47e1-8451-ef13a9a2d339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:26.485911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5ad83957-d8bd-47e1-8451-ef13a9a2d339 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:26.487067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5ad83957-d8bd-47e1-8451-ef13a9a2d339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:26.520719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:26.551487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 418/841] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:26.820666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40ced-a54d-46b7-8f60-f39d4ea10877 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:26.916581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40ced-a54d-46b7-8f60-f39d4ea10877 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:26.932310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:26.949445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24d8c0-c7b3-489b-9fee-f5835ed0695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:26.950041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24d8c0-c7b3-489b-9fee-f5835ed0695b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:26.950312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24d8c0-c7b3-489b-9fee-f5835ed0695b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:26.950570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24d8c0-c7b3-489b-9fee-f5835ed0695b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:26.951635 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24d8c0-c7b3-489b-9fee-f5835ed0695b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:26.953051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24d8c0-c7b3-489b-9fee-f5835ed0695b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:26.954537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 424/842] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:26.960747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad83957-d8bd-47e1-8451-ef13a9a2d339 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:27.054204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad83957-d8bd-47e1-8451-ef13a9a2d339 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:27.086739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6482a7-646a-47a6-b124-33899a2f7b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:27.087270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6482a7-646a-47a6-b124-33899a2f7b66 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:27.087545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6482a7-646a-47a6-b124-33899a2f7b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:27.087813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6482a7-646a-47a6-b124-33899a2f7b66 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:27.088479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6482a7-646a-47a6-b124-33899a2f7b66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:27.089873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6482a7-646a-47a6-b124-33899a2f7b66 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:27.110224 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 419/843] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:12:26 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:29.243354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f349707-7c1d-4e58-b33f-e1bb2ec39c01 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:29.243875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f349707-7c1d-4e58-b33f-e1bb2ec39c01 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:29.243875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f349707-7c1d-4e58-b33f-e1bb2ec39c01 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:29.245197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 425/844] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:29.250186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b7269-fc4d-4eed-b939-94c349aa5762 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:29.250545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b7269-fc4d-4eed-b939-94c349aa5762 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:29.250728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b7269-fc4d-4eed-b939-94c349aa5762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:29.269162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-928b7269-fc4d-4eed-b939-94c349aa5762 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:29.270299 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-928b7269-fc4d-4eed-b939-94c349aa5762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:29.292355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:29.334799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 420/845] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:29.604861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f349707-7c1d-4e58-b33f-e1bb2ec39c01 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:29.700468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f349707-7c1d-4e58-b33f-e1bb2ec39c01 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:29.732782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:29.733390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:29.733657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:29.733919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] PATH_INFO: `/v3/roles/fc711bb446524efbbb3ee062f6ff2702` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:29.735255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] RBAC: Authorizing `identity:get_role(role_id=fc711bb446524efbbb3ee062f6ff2702)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:29.736621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:29.737893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 426/846] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:29 2022] GET /identity/v3/roles/fc711bb446524efbbb3ee062f6ff2702 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:29.743694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b7269-fc4d-4eed-b939-94c349aa5762 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:29.837808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b7269-fc4d-4eed-b939-94c349aa5762 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:29.870023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:29.870636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:29.870906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:29.871168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] PATH_INFO: `/v3/users/dd921ca25c314d1da581af3343a9d0b0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:29.883600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=dd921ca25c314d1da581af3343a9d0b0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:29.885113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:29.886660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 421/847] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:29 2022] GET /identity/v3/users/dd921ca25c314d1da581af3343a9d0b0 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:29.892610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:29.942407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:29.986863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac82082-6a33-4263-9f28-b7b9aa15a35c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:30.018279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d5e8a-907b-4812-a4ef-66d9bc13be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:30.018884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d5e8a-907b-4812-a4ef-66d9bc13be4d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:30.019156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d5e8a-907b-4812-a4ef-66d9bc13be4d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:30.019421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d5e8a-907b-4812-a4ef-66d9bc13be4d None admin] PATH_INFO: `/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:30.020526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d5e8a-907b-4812-a4ef-66d9bc13be4d None admin] RBAC: Authorizing `identity:get_project(project_id=4b6a7621bc2447c78cf9267f8c5c2a2d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:30.022494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d5e8a-907b-4812-a4ef-66d9bc13be4d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:30.024061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 427/848] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:12:29 2022] GET /identity/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:30.029929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:30.047396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:30.125327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3061a88-2aeb-4621-9284-5ec852f53bd4 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:30.156757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406923b-fd26-4d16-99c3-d73eb8fe612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:30.157388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406923b-fd26-4d16-99c3-d73eb8fe612a None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:30.157659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406923b-fd26-4d16-99c3-d73eb8fe612a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:30.157921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406923b-fd26-4d16-99c3-d73eb8fe612a None admin] PATH_INFO: `/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d/users/dd921ca25c314d1da581af3343a9d0b0/roles/fc711bb446524efbbb3ee062f6ff2702` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:30.159974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406923b-fd26-4d16-99c3-d73eb8fe612a None admin] RBAC: Authorizing `identity:create_grant(project_id=4b6a7621bc2447c78cf9267f8c5c2a2d, user_id=dd921ca25c314d1da581af3343a9d0b0, role_id=fc711bb446524efbbb3ee062f6ff2702)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:30.161403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406923b-fd26-4d16-99c3-d73eb8fe612a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:30.174195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 422/849] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:12:30 2022] PUT /identity/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d/users/dd921ca25c314d1da581af3343a9d0b0/roles/fc711bb446524efbbb3ee062f6ff2702 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:12:32.256839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1ab1-0d3e-4879-9755-1242f8d6e4aa None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:32.257359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1ab1-0d3e-4879-9755-1242f8d6e4aa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:32.257359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1ab1-0d3e-4879-9755-1242f8d6e4aa None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:32.258659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 428/850] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:32.263892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7b77-21f6-4f78-a181-2c7953fbcd56 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:32.264329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7b77-21f6-4f78-a181-2c7953fbcd56 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:32.264466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7b77-21f6-4f78-a181-2c7953fbcd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:32.282116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-429b7b77-21f6-4f78-a181-2c7953fbcd56 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:32.283258 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-429b7b77-21f6-4f78-a181-2c7953fbcd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:32.345419 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 423/851] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:32.609819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1ab1-0d3e-4879-9755-1242f8d6e4aa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:32.706841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1ab1-0d3e-4879-9755-1242f8d6e4aa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:32.725441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:32.738765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:32.739372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:32.739642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:32.739907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] PATH_INFO: `/v3/roles/fc711bb446524efbbb3ee062f6ff2702` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:32.741792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] RBAC: Authorizing `identity:get_role(role_id=fc711bb446524efbbb3ee062f6ff2702)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:32.743121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:32.744409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 429/852] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:32 2022] GET /identity/v3/roles/fc711bb446524efbbb3ee062f6ff2702 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:32.750248 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b7b77-21f6-4f78-a181-2c7953fbcd56 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:32.758708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:32.844390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b7b77-21f6-4f78-a181-2c7953fbcd56 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:32.876760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:32.877375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:32.877649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:32.877916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] PATH_INFO: `/v3/users/dd921ca25c314d1da581af3343a9d0b0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:32.879009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] RBAC: Authorizing `identity:get_user(user_id=dd921ca25c314d1da581af3343a9d0b0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:32.880444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:32.882575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 424/853] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:32 2022] GET /identity/v3/users/dd921ca25c314d1da581af3343a9d0b0 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:32.888240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:32.981211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c84ab-3a4c-449e-b85e-cab7ff64f087 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:33.013089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:33.013772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:33.014044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:33.014310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:33.065438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:33.077841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:33.079273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:33.173510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 430/854] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:12:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:12:33.179978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:33.250764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:33.275227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b17f19-cf03-4822-9732-bdf5a0dc97d0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:33.309021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c561d-37c1-4e51-a3f0-f84e1e36987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:33.309627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c561d-37c1-4e51-a3f0-f84e1e36987a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:33.309900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c561d-37c1-4e51-a3f0-f84e1e36987a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:33.310165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c561d-37c1-4e51-a3f0-f84e1e36987a None admin] PATH_INFO: `/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:33.311248 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c561d-37c1-4e51-a3f0-f84e1e36987a None admin] RBAC: Authorizing `identity:get_project(project_id=4b6a7621bc2447c78cf9267f8c5c2a2d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:33.312671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c561d-37c1-4e51-a3f0-f84e1e36987a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:33.314243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 425/855] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:12:33 2022] GET /identity/v3/projects/4b6a7621bc2447c78cf9267f8c5c2a2d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:33.320415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:33.403328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:33.415630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc4de2-9f47-4f56-9f84-39b007a24ebb None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:33.447424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d310592-dd97-4160-a38a-0b23dc5b411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:33.447955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d310592-dd97-4160-a38a-0b23dc5b411f None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:33.448252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d310592-dd97-4160-a38a-0b23dc5b411f None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:33.448520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d310592-dd97-4160-a38a-0b23dc5b411f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:33.449221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d310592-dd97-4160-a38a-0b23dc5b411f None admin] RBAC: Adding query filter params (role.id=fc711bb446524efbbb3ee062f6ff2702, scope.project.id=4b6a7621bc2447c78cf9267f8c5c2a2d, user.id=dd921ca25c314d1da581af3343a9d0b0) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:33.449518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d310592-dd97-4160-a38a-0b23dc5b411f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:33.450907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d310592-dd97-4160-a38a-0b23dc5b411f None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:33.456365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 431/856] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:12:33 2022] GET /identity/v3/role_assignments?user.id=dd921ca25c314d1da581af3343a9d0b0&scope.project.id=4b6a7621bc2447c78cf9267f8c5c2a2d&role.id=fc711bb446524efbbb3ee062f6ff2702 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:12:35.576449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1dd34-728e-417b-b9d7-fdf58851c3c6 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:35.577000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1dd34-728e-417b-b9d7-fdf58851c3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:35.577000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1dd34-728e-417b-b9d7-fdf58851c3c6 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:35.578284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 426/857] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:35.583394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded58dbd-70d1-4710-888a-c7969f6b4508 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:35.583790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded58dbd-70d1-4710-888a-c7969f6b4508 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:35.584510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded58dbd-70d1-4710-888a-c7969f6b4508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:35.602157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ded58dbd-70d1-4710-888a-c7969f6b4508 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:35.603308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ded58dbd-70d1-4710-888a-c7969f6b4508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:35.666651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 432/858] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:35.935763 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1dd34-728e-417b-b9d7-fdf58851c3c6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:36.032896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1dd34-728e-417b-b9d7-fdf58851c3c6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:36.065788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ebe4-71ba-4eae-819a-ea034b1e717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:36.066393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ebe4-71ba-4eae-819a-ea034b1e717e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:36.066667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ebe4-71ba-4eae-819a-ea034b1e717e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:36.066934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ebe4-71ba-4eae-819a-ea034b1e717e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:36.068027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ebe4-71ba-4eae-819a-ea034b1e717e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:36.069489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ebe4-71ba-4eae-819a-ea034b1e717e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:36.071002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 427/859] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:36.076649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded58dbd-70d1-4710-888a-c7969f6b4508 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:36.153556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:36.171683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded58dbd-70d1-4710-888a-c7969f6b4508 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:36.204325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60505949-201d-407b-8a06-1b587b81066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:36.204862 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60505949-201d-407b-8a06-1b587b81066b None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:36.205134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60505949-201d-407b-8a06-1b587b81066b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:36.205400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60505949-201d-407b-8a06-1b587b81066b None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:36.206036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60505949-201d-407b-8a06-1b587b81066b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:36.207430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60505949-201d-407b-8a06-1b587b81066b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:36.232534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 433/860] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:12:36 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:12:38.359233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758064-0c20-4a5f-81ab-41ac1ccf9f00 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:38.359778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758064-0c20-4a5f-81ab-41ac1ccf9f00 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:38.359778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758064-0c20-4a5f-81ab-41ac1ccf9f00 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:38.361101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 428/861] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:38.366872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec996c-7e07-48fd-af74-fb37cd38f51d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:38.367261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec996c-7e07-48fd-af74-fb37cd38f51d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:38.367400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec996c-7e07-48fd-af74-fb37cd38f51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:38.384926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-52ec996c-7e07-48fd-af74-fb37cd38f51d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:38.386070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-52ec996c-7e07-48fd-af74-fb37cd38f51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:38.448236 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 434/862] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:38.714917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758064-0c20-4a5f-81ab-41ac1ccf9f00 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:38.726889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:38.811679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758064-0c20-4a5f-81ab-41ac1ccf9f00 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:38.843537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfaca1-a049-4ced-83e0-9e68f8636639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:38.844136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfaca1-a049-4ced-83e0-9e68f8636639 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:38.844429 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfaca1-a049-4ced-83e0-9e68f8636639 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:38.844694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfaca1-a049-4ced-83e0-9e68f8636639 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:38.845768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfaca1-a049-4ced-83e0-9e68f8636639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:38.847161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfaca1-a049-4ced-83e0-9e68f8636639 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:38.848692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 429/863] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:12:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:38.854286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec996c-7e07-48fd-af74-fb37cd38f51d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:38.913143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:38.950035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec996c-7e07-48fd-af74-fb37cd38f51d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:38.982053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743cf77-8ccd-45c8-adb4-9044d3e92e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:38.982640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743cf77-8ccd-45c8-adb4-9044d3e92e39 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:38.982912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743cf77-8ccd-45c8-adb4-9044d3e92e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:38.983180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743cf77-8ccd-45c8-adb4-9044d3e92e39 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:38.983821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743cf77-8ccd-45c8-adb4-9044d3e92e39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:38.985263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743cf77-8ccd-45c8-adb4-9044d3e92e39 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:39.006298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 435/864] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:12:38 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:12:41.163132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bd69-dfc1-48e6-b597-9a2d5add2719 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:41.163643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bd69-dfc1-48e6-b597-9a2d5add2719 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:41.163734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bd69-dfc1-48e6-b597-9a2d5add2719 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:41.165025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 430/865] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:41.170065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972b2ca-8484-4ef0-a063-f066909e8861 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:41.170440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972b2ca-8484-4ef0-a063-f066909e8861 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:41.170607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972b2ca-8484-4ef0-a063-f066909e8861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:41.188762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0972b2ca-8484-4ef0-a063-f066909e8861 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:41.189922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0972b2ca-8484-4ef0-a063-f066909e8861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:41.254037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 436/866] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:41.519073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bd69-dfc1-48e6-b597-9a2d5add2719 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:41.601339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:41.624030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bd69-dfc1-48e6-b597-9a2d5add2719 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:41.658413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:41.659023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:41.659300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:41.659571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:41.669309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:41.674094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:41.675142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:41.676408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 431/867] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:12:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:41.681709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972b2ca-8484-4ef0-a063-f066909e8861 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:41.728214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:41.777110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972b2ca-8484-4ef0-a063-f066909e8861 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:41.809209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:41.809791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:41.810063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:41.810327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:41.810972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:41.811266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:41.812633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:41.820911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 437/868] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:41.827136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:41.920452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f49570-a862-40a8-9fa4-82d6222f6ddc None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:41.952669 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:41.953267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:41.953538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:41.953803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] PATH_INFO: `/v3/users/8544e275d7f74803849e8288d31fcdac` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:41.960634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:41.966416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] RBAC: Authorizing `identity:get_user(user_id=8544e275d7f74803849e8288d31fcdac)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:41.967813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:41.969381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 432/869] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:41 2022] GET /identity/v3/users/8544e275d7f74803849e8288d31fcdac => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:41.975459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:42.069354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0550b-d9a4-402e-bb57-4327e44a7c0c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:42.081180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:42.101812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a433b4-fb29-4e05-ba54-7c93ae670d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:42.102427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a433b4-fb29-4e05-ba54-7c93ae670d6b None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:42.102701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a433b4-fb29-4e05-ba54-7c93ae670d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:42.102967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a433b4-fb29-4e05-ba54-7c93ae670d6b None admin] PATH_INFO: `/v3/projects/8c5491b51dd64c59b87fffe85c601abb` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:42.104059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a433b4-fb29-4e05-ba54-7c93ae670d6b None admin] RBAC: Authorizing `identity:get_project(project_id=8c5491b51dd64c59b87fffe85c601abb)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:42.105489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a433b4-fb29-4e05-ba54-7c93ae670d6b None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:42.107062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 438/870] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:12:41 2022] GET /identity/v3/projects/8c5491b51dd64c59b87fffe85c601abb => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:42.112966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:42.208334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5ba35-0ee6-4f09-9a27-384ddedf382e None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:42.241102 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed56dda-e5aa-4b9a-bec1-6f660a446381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:42.241732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed56dda-e5aa-4b9a-bec1-6f660a446381 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:42.242002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed56dda-e5aa-4b9a-bec1-6f660a446381 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:42.242269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed56dda-e5aa-4b9a-bec1-6f660a446381 None admin] PATH_INFO: `/v3/projects/8c5491b51dd64c59b87fffe85c601abb/users/8544e275d7f74803849e8288d31fcdac/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:42.244112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed56dda-e5aa-4b9a-bec1-6f660a446381 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c5491b51dd64c59b87fffe85c601abb, user_id=8544e275d7f74803849e8288d31fcdac, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:42.245546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed56dda-e5aa-4b9a-bec1-6f660a446381 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:42.258124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 433/871] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:12:42 2022] PUT /identity/v3/projects/8c5491b51dd64c59b87fffe85c601abb/users/8544e275d7f74803849e8288d31fcdac/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:12:44.357946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9186ef-45ac-488a-9f1d-57846e608237 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:44.358530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9186ef-45ac-488a-9f1d-57846e608237 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:44.358530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9186ef-45ac-488a-9f1d-57846e608237 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:44.359796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 439/872] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:44.365708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5836-79c3-4abb-83e5-f5a97b76213d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:44.366143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5836-79c3-4abb-83e5-f5a97b76213d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:44.366248 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5836-79c3-4abb-83e5-f5a97b76213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:44.383762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7dbd5836-79c3-4abb-83e5-f5a97b76213d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:44.384944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7dbd5836-79c3-4abb-83e5-f5a97b76213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:44.432859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:44.449539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 434/873] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:44.722214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9186ef-45ac-488a-9f1d-57846e608237 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:44.819137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9186ef-45ac-488a-9f1d-57846e608237 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:44.851599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:44.852203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:44.852506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:44.852772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:44.862406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:44.867220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:44.868313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:44.869307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 440/874] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:12:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:44.874139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5836-79c3-4abb-83e5-f5a97b76213d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:44.967959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5836-79c3-4abb-83e5-f5a97b76213d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:44.999259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:44.999845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:45.000121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:45.000415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:45.001063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:45.001359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:45.002698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:45.010780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 435/875] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:45.016720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:45.057281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:45.113092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb28d9-6a0c-476a-8a37-6f45d76aefc4 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:45.146627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:45.147181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:45.147457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:45.147722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] PATH_INFO: `/v3/users/8544e275d7f74803849e8288d31fcdac` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:45.148828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] RBAC: Authorizing `identity:get_user(user_id=8544e275d7f74803849e8288d31fcdac)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:45.150228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:45.151757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 441/876] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:45 2022] GET /identity/v3/users/8544e275d7f74803849e8288d31fcdac => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:45.157445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:45.203798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:45.252563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55a44-df21-4692-aaf0-f1d618d11f32 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:45.284977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:45.285557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:45.285832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:45.286099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:45.348934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:45.350346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:45.395341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:45.445186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 436/877] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:12:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:12:45.451001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:45.545749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6183cae-f612-483b-8abb-b97db5069c48 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:45.554512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:45.582885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c90c9-c68f-44a7-8c8e-8818060be249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:45.583523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c90c9-c68f-44a7-8c8e-8818060be249 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:45.583795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c90c9-c68f-44a7-8c8e-8818060be249 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:45.584061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c90c9-c68f-44a7-8c8e-8818060be249 None admin] PATH_INFO: `/v3/projects/8c5491b51dd64c59b87fffe85c601abb` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:45.585179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c90c9-c68f-44a7-8c8e-8818060be249 None admin] RBAC: Authorizing `identity:get_project(project_id=8c5491b51dd64c59b87fffe85c601abb)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:45.586585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c90c9-c68f-44a7-8c8e-8818060be249 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:45.588174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 442/878] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:12:45 2022] GET /identity/v3/projects/8c5491b51dd64c59b87fffe85c601abb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:45.594269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:45.688960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29277076-2b6a-4c3c-a5ac-59abb054d8a2 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:45.722194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d9a7c-487b-41b9-9f2e-e81a5af08901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:45.722729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d9a7c-487b-41b9-9f2e-e81a5af08901 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:45.723002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d9a7c-487b-41b9-9f2e-e81a5af08901 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:45.723268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d9a7c-487b-41b9-9f2e-e81a5af08901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:45.724513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d9a7c-487b-41b9-9f2e-e81a5af08901 None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=8c5491b51dd64c59b87fffe85c601abb, user.id=8544e275d7f74803849e8288d31fcdac) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:45.724817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d9a7c-487b-41b9-9f2e-e81a5af08901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:45.726212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d9a7c-487b-41b9-9f2e-e81a5af08901 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:45.731731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 437/879] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:12:45 2022] GET /identity/v3/role_assignments?user.id=8544e275d7f74803849e8288d31fcdac&scope.project.id=8c5491b51dd64c59b87fffe85c601abb&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:47.841525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7c968-cf42-4600-8b13-a3488d517d22 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:47.842040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7c968-cf42-4600-8b13-a3488d517d22 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:47.842040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7c968-cf42-4600-8b13-a3488d517d22 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:47.843365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 443/880] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:47.848601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0e62d-f720-40df-8c05-8c1d7b472942 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:47.849028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0e62d-f720-40df-8c05-8c1d7b472942 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:47.849172 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0e62d-f720-40df-8c05-8c1d7b472942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:47.868035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-aaf0e62d-f720-40df-8c05-8c1d7b472942 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:47.869194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-aaf0e62d-f720-40df-8c05-8c1d7b472942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:47.876183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:47.932526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 438/881] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:48.201661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7c968-cf42-4600-8b13-a3488d517d22 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:48.298185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7c968-cf42-4600-8b13-a3488d517d22 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:48.329987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da492e40-f54c-437e-98db-a56936d45962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:48.330577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da492e40-f54c-437e-98db-a56936d45962 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:48.330849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da492e40-f54c-437e-98db-a56936d45962 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:48.331114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da492e40-f54c-437e-98db-a56936d45962 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:48.331755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da492e40-f54c-437e-98db-a56936d45962 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:48.333622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da492e40-f54c-437e-98db-a56936d45962 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:48.348389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 444/882] 10.222.0.48 () {60 vars in 1138 bytes} [Thu Oct 27 13:12:48 2022] POST /identity/v3/domains => generated 267 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:50.499002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6612c-7719-45e8-96cf-cb06d4b0888c None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:50.499612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6612c-7719-45e8-96cf-cb06d4b0888c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:50.499612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6612c-7719-45e8-96cf-cb06d4b0888c None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:50.500892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 439/883] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:50.506181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab428f64-8113-4ae3-bb3a-6a1e89a41dcd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:50.506602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab428f64-8113-4ae3-bb3a-6a1e89a41dcd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:50.506727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab428f64-8113-4ae3-bb3a-6a1e89a41dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:50.524525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ab428f64-8113-4ae3-bb3a-6a1e89a41dcd None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:50.525689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ab428f64-8113-4ae3-bb3a-6a1e89a41dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:50.553514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:50.590354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 445/884] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:50.857455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6612c-7719-45e8-96cf-cb06d4b0888c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:50.953505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6612c-7719-45e8-96cf-cb06d4b0888c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:50.985201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739667b-9902-4220-a54d-a64650614932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:50.985798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739667b-9902-4220-a54d-a64650614932 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:50.986069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739667b-9902-4220-a54d-a64650614932 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:50.986335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739667b-9902-4220-a54d-a64650614932 None admin] PATH_INFO: `/v3/domains/9bc866bfae0247b1b5bbbdb1448eb6b6` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:50.996867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739667b-9902-4220-a54d-a64650614932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9bc866bfae0247b1b5bbbdb1448eb6b6)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:50.998261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739667b-9902-4220-a54d-a64650614932 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:50.999776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 440/885] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Oct 27 13:12:50 2022] GET /identity/v3/domains/9bc866bfae0247b1b5bbbdb1448eb6b6 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:51.005373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab428f64-8113-4ae3-bb3a-6a1e89a41dcd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:51.098213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab428f64-8113-4ae3-bb3a-6a1e89a41dcd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:51.130970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33047a-c496-4c41-83d7-7c052faf6b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:51.131557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33047a-c496-4c41-83d7-7c052faf6b6e None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:51.131831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33047a-c496-4c41-83d7-7c052faf6b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:51.132097 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33047a-c496-4c41-83d7-7c052faf6b6e None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:51.132755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33047a-c496-4c41-83d7-7c052faf6b6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:51.134150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33047a-c496-4c41-83d7-7c052faf6b6e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:51.159815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 446/886] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:12:51 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:53.324017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830373a2-fd8a-4baa-b613-e27915310f91 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:53.324601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830373a2-fd8a-4baa-b613-e27915310f91 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:53.324601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830373a2-fd8a-4baa-b613-e27915310f91 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:53.325902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 441/887] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:53.331198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011945e-57f4-4481-88d8-ff6a728da556 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:53.331622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011945e-57f4-4481-88d8-ff6a728da556 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:53.331808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011945e-57f4-4481-88d8-ff6a728da556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:53.350049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c011945e-57f4-4481-88d8-ff6a728da556 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:53.351214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c011945e-57f4-4481-88d8-ff6a728da556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:53.363840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:53.416437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 447/888] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:53.686895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830373a2-fd8a-4baa-b613-e27915310f91 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:53.689812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:53.785128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830373a2-fd8a-4baa-b613-e27915310f91 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:53.816570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a739e3d-03a9-471e-8dc1-135dd8f76f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:53.817747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a739e3d-03a9-471e-8dc1-135dd8f76f77 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:53.818021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a739e3d-03a9-471e-8dc1-135dd8f76f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:53.818284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a739e3d-03a9-471e-8dc1-135dd8f76f77 None admin] PATH_INFO: `/v3/domains/9bc866bfae0247b1b5bbbdb1448eb6b6` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:53.819362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a739e3d-03a9-471e-8dc1-135dd8f76f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9bc866bfae0247b1b5bbbdb1448eb6b6)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:53.820803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a739e3d-03a9-471e-8dc1-135dd8f76f77 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:53.822323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 442/889] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Oct 27 13:12:53 2022] GET /identity/v3/domains/9bc866bfae0247b1b5bbbdb1448eb6b6 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:53.828039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011945e-57f4-4481-88d8-ff6a728da556 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:53.921796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011945e-57f4-4481-88d8-ff6a728da556 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:53.954441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7708cf-827e-49f0-9c25-64fa4101a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:53.955025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7708cf-827e-49f0-9c25-64fa4101a246 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:53.955300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7708cf-827e-49f0-9c25-64fa4101a246 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:53.955568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7708cf-827e-49f0-9c25-64fa4101a246 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:53.956208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7708cf-827e-49f0-9c25-64fa4101a246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:53.957641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7708cf-827e-49f0-9c25-64fa4101a246 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:53.977782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 448/890] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:12:53 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:12:56.131278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40eacb1-1d2e-4c9a-8680-cc127ba33596 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:56.131807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40eacb1-1d2e-4c9a-8680-cc127ba33596 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:56.131807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40eacb1-1d2e-4c9a-8680-cc127ba33596 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:56.133158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 443/891] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:56.138380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddcefb-31b9-4a70-a412-96b90fc67f9d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:56.138781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddcefb-31b9-4a70-a412-96b90fc67f9d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:56.138927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddcefb-31b9-4a70-a412-96b90fc67f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:56.146861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:56.158833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-89ddcefb-31b9-4a70-a412-96b90fc67f9d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:56.159998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-89ddcefb-31b9-4a70-a412-96b90fc67f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:56.223890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 449/892] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:12:56.489286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40eacb1-1d2e-4c9a-8680-cc127ba33596 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:56.556034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:56.587711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40eacb1-1d2e-4c9a-8680-cc127ba33596 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:56.622284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:56.622896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:56.623167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:56.623430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:56.634911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:56.639552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:56.640722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:12:56.642213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 444/893] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:12:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:56.646930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddcefb-31b9-4a70-a412-96b90fc67f9d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:56.740068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddcefb-31b9-4a70-a412-96b90fc67f9d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:56.772697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:56.773292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:56.773567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:56.773833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:56.774480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:56.774777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:56.776123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:56.777530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:56.784911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 450/894] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:12:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:56.790650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:56.885197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109433c-95a9-4313-8ca0-0f98b96cffff None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:56.911525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:56.917486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:56.918085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:56.918357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:56.918621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] PATH_INFO: `/v3/users/8abad4bd83ab43439ee7f1345a77bc04` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:56.931568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] RBAC: Authorizing `identity:get_user(user_id=8abad4bd83ab43439ee7f1345a77bc04)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:56.933003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:56.934628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 445/895] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:56 2022] GET /identity/v3/users/8abad4bd83ab43439ee7f1345a77bc04 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:56.940223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:57.035058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bd0bb-ee2f-4f60-8288-f23e4250f390 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:57.067223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08576ecf-3a08-4c40-abdf-56d652bb35c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:57.067830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08576ecf-3a08-4c40-abdf-56d652bb35c4 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:57.068108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08576ecf-3a08-4c40-abdf-56d652bb35c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:57.068408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08576ecf-3a08-4c40-abdf-56d652bb35c4 None admin] PATH_INFO: `/v3/projects/687af7dfda674877bb2e122c34c079cf` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:57.069508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08576ecf-3a08-4c40-abdf-56d652bb35c4 None admin] RBAC: Authorizing `identity:get_project(project_id=687af7dfda674877bb2e122c34c079cf)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:57.070908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08576ecf-3a08-4c40-abdf-56d652bb35c4 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:57.072511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 451/896] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:12:56 2022] GET /identity/v3/projects/687af7dfda674877bb2e122c34c079cf => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:57.078142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:57.172686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281c9e-d375-40f9-8a03-0e4c70070f76 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:57.204648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840cb4a-6c22-4c6a-a6e7-11b38703a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:57.205269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840cb4a-6c22-4c6a-a6e7-11b38703a022 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:57.205540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840cb4a-6c22-4c6a-a6e7-11b38703a022 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:57.205801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840cb4a-6c22-4c6a-a6e7-11b38703a022 None admin] PATH_INFO: `/v3/projects/687af7dfda674877bb2e122c34c079cf/users/8abad4bd83ab43439ee7f1345a77bc04/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:57.207670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840cb4a-6c22-4c6a-a6e7-11b38703a022 None admin] RBAC: Authorizing `identity:create_grant(project_id=687af7dfda674877bb2e122c34c079cf, user_id=8abad4bd83ab43439ee7f1345a77bc04, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:57.209091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840cb4a-6c22-4c6a-a6e7-11b38703a022 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:57.221312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 446/897] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:12:57 2022] PUT /identity/v3/projects/687af7dfda674877bb2e122c34c079cf/users/8abad4bd83ab43439ee7f1345a77bc04/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:12:59.335238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2add4a-3c0e-4492-a98f-332c06fd22dc None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:59.335783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2add4a-3c0e-4492-a98f-332c06fd22dc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:59.335783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2add4a-3c0e-4492-a98f-332c06fd22dc None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:59.337110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 452/898] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:12:59.342349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32abcb3-df2d-4db7-8209-f11e02f94a9f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:59.342760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32abcb3-df2d-4db7-8209-f11e02f94a9f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:59.342884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32abcb3-df2d-4db7-8209-f11e02f94a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:59.360941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d32abcb3-df2d-4db7-8209-f11e02f94a9f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:12:59.362095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d32abcb3-df2d-4db7-8209-f11e02f94a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:12:59.404587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:59.427176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 447/899] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:12:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:12:59.693720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2add4a-3c0e-4492-a98f-332c06fd22dc None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:59.753191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:12:59.791552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2add4a-3c0e-4492-a98f-332c06fd22dc None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:59.825303 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:59.825921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:59.826197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:59.826467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:59.836198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:12:59.840933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:12:59.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:12:59.842984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 453/900] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:12:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:12:59.848158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32abcb3-df2d-4db7-8209-f11e02f94a9f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:12:59.943532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32abcb3-df2d-4db7-8209-f11e02f94a9f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:12:59.975926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:12:59.976550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:12:59.976828 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:12:59.977095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:12:59.977744 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:12:59.978039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:12:59.979386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:12:59.987017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 448/901] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:12:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:12:59.993320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:00.087387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a97e5-8124-4e40-bef3-323ada72950a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:00.109811 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:00.120954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:00.121552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:00.121821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:00.122083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] PATH_INFO: `/v3/users/8abad4bd83ab43439ee7f1345a77bc04` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:00.123152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] RBAC: Authorizing `identity:get_user(user_id=8abad4bd83ab43439ee7f1345a77bc04)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:00.124569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:00.126077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 454/902] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Oct 27 13:12:59 2022] GET /identity/v3/users/8abad4bd83ab43439ee7f1345a77bc04 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:13:00.131695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:00.172049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:00.227501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f888ee4-8637-416b-b217-37802ba0046b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:00.259025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:00.259606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:00.259876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:00.260139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:00.323214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:00.324651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:00.362411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:00.418275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 449/903] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:13:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:13:00.424050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:00.517200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b32fb-c6a5-45c5-8cc0-e4cba7396202 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:00.548753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb76e5-586e-483e-87b8-ed497b9a8b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:00.549349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb76e5-586e-483e-87b8-ed497b9a8b94 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:00.549614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb76e5-586e-483e-87b8-ed497b9a8b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:00.549875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb76e5-586e-483e-87b8-ed497b9a8b94 None admin] PATH_INFO: `/v3/projects/687af7dfda674877bb2e122c34c079cf` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:00.550939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb76e5-586e-483e-87b8-ed497b9a8b94 None admin] RBAC: Authorizing `identity:get_project(project_id=687af7dfda674877bb2e122c34c079cf)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:00.552396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb76e5-586e-483e-87b8-ed497b9a8b94 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:00.553941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 455/904] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Oct 27 13:13:00 2022] GET /identity/v3/projects/687af7dfda674877bb2e122c34c079cf => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:13:00.559814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:00.655823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaea26b-b0c4-4b14-9b10-71ac7f0a38be None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:00.689697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d47c5-5cdb-47f8-8546-fea0d05a57a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:00.690285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d47c5-5cdb-47f8-8546-fea0d05a57a6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:00.690561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d47c5-5cdb-47f8-8546-fea0d05a57a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:00.690830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d47c5-5cdb-47f8-8546-fea0d05a57a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:00.691539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d47c5-5cdb-47f8-8546-fea0d05a57a6 None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=687af7dfda674877bb2e122c34c079cf, user.id=8abad4bd83ab43439ee7f1345a77bc04) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:13:00.692496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d47c5-5cdb-47f8-8546-fea0d05a57a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:00.693910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d47c5-5cdb-47f8-8546-fea0d05a57a6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:00.699453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 450/905] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:13:00 2022] GET /identity/v3/role_assignments?user.id=8abad4bd83ab43439ee7f1345a77bc04&scope.project.id=687af7dfda674877bb2e122c34c079cf&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:13:10.544864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=70 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:10.696922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:55.704772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6481a46-ec09-487c-9914-269cef38ed2c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:55.705337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6481a46-ec09-487c-9914-269cef38ed2c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:55.705337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6481a46-ec09-487c-9914-269cef38ed2c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:55.706635 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 456/906] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:13:55.712165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f89ffe-a635-42c3-a491-88aaa23e45ee None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:55.712795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f89ffe-a635-42c3-a491-88aaa23e45ee None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:55.712795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f89ffe-a635-42c3-a491-88aaa23e45ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:55.734104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d7f89ffe-a635-42c3-a491-88aaa23e45ee None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:13:55.735289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d7f89ffe-a635-42c3-a491-88aaa23e45ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:13:55.836274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 451/907] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:13:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:13:56.111695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6481a46-ec09-487c-9914-269cef38ed2c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:56.243218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6481a46-ec09-487c-9914-269cef38ed2c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:56.283239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:56.283863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:56.284185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:56.284489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:56.298828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:56.300432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:13:56.315708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:13:56.316833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:13:56.321442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:13:56.321442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:13:56.321442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:13:56.321442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 457/908] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:13:56.324306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f89ffe-a635-42c3-a491-88aaa23e45ee None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:56.449004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f89ffe-a635-42c3-a491-88aaa23e45ee None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:56.478745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:56.486341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f1b37-3623-4911-9fe7-8a100b83fd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:56.487005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1b37-3623-4911-9fe7-8a100b83fd99 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:56.487251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1b37-3623-4911-9fe7-8a100b83fd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:56.487527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1b37-3623-4911-9fe7-8a100b83fd99 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:56.488187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f1b37-3623-4911-9fe7-8a100b83fd99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:13:56.488488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f1b37-3623-4911-9fe7-8a100b83fd99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:56.489825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f1b37-3623-4911-9fe7-8a100b83fd99 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:56.510000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 452/909] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:13:56.515824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:56.581353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:56.651111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fe980-13f1-4d6e-ae25-7223361c8c3a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:56.689396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaa81b-cec3-4d42-b64d-98fda35de5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:56.690381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaa81b-cec3-4d42-b64d-98fda35de5d3 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:56.690693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaa81b-cec3-4d42-b64d-98fda35de5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:56.690969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaa81b-cec3-4d42-b64d-98fda35de5d3 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:56.691719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaa81b-cec3-4d42-b64d-98fda35de5d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:56.693262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaa81b-cec3-4d42-b64d-98fda35de5d3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:56.732847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 458/910] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Oct 27 13:13:56 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:13:59.108263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c95f9b-65f3-42ca-b923-8a6e0bd98c31 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:59.108812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c95f9b-65f3-42ca-b923-8a6e0bd98c31 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:59.108812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c95f9b-65f3-42ca-b923-8a6e0bd98c31 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:59.110795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 453/911] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:13:59.116149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a6df9-bdbf-4a6a-967d-19d9fb39caaa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:59.116690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a6df9-bdbf-4a6a-967d-19d9fb39caaa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:59.116690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a6df9-bdbf-4a6a-967d-19d9fb39caaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:59.137158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5b9a6df9-bdbf-4a6a-967d-19d9fb39caaa None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:13:59.138462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5b9a6df9-bdbf-4a6a-967d-19d9fb39caaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:13:59.218126 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 459/912] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:13:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:13:59.506727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c95f9b-65f3-42ca-b923-8a6e0bd98c31 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:59.621543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c95f9b-65f3-42ca-b923-8a6e0bd98c31 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:59.662082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:59.662743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:59.663098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:59.663361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:59.675021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:13:59.680195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:13:59.681271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:13:59.682090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 454/913] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:13:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:13:59.687604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a6df9-bdbf-4a6a-967d-19d9fb39caaa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:59.729872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:59.795773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a6df9-bdbf-4a6a-967d-19d9fb39caaa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:13:59.833351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:13:59.833982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:13:59.834258 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:13:59.834524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:13:59.835180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:13:59.835503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:13:59.837144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:13:59.846583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 460/914] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:13:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:13:59.852933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:13:59.910126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:13:59.987775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a63ed0-17d6-44fa-9cb2-55b854d12b38 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:00.027670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:00.028724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:00.029015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:00.029273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:00.042877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:00.044561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:00.063980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:00.065061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:00.066348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:00.066348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:00.066348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:00.071710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 455/915] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:00.071839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:00.182595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:00.186796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2590b59-ddb1-4574-932c-ce46f5cd48bc None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:00.225041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:00.225678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:00.225951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:00.226245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:00.228061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:00.228418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:00.229847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:00.244048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 461/916] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:00.256285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:00.371357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42874034-44b1-4b9c-bbf4-e3fc4907482c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:00.373249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:00.410006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:00.410730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:00.410992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:00.411306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:00.420002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:00.421577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:14:00.434518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:00.435477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:00.442113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 456/917] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:00.444414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:00.555729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304aaca-39cc-48b7-aac9-380a6ae5f0f3 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:00.593129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20161c50-9702-453a-88b6-46493215e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:00.593738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20161c50-9702-453a-88b6-46493215e433 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:00.594072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20161c50-9702-453a-88b6-46493215e433 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:00.594366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20161c50-9702-453a-88b6-46493215e433 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:00.595060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20161c50-9702-453a-88b6-46493215e433 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:00.595373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20161c50-9702-453a-88b6-46493215e433 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:00.597539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20161c50-9702-453a-88b6-46493215e433 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:00.616086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 462/918] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Oct 27 13:14:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:00.622136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:00.730831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e0da-1e28-4a0e-8747-71f3d1c084d8 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:00.766708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:00.767338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:00.767615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:00.767884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:00.778436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:00.779888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:00.794425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:00.795584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:00.796526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:00.796526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:00.796526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:00.796526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 457/919] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 27 13:14:00.802363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20161c50-9702-453a-88b6-46493215e433 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:00.821462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:00.903855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20161c50-9702-453a-88b6-46493215e433 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:00.941013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:00.941616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:00.941955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:00.942153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:00.942864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:00.943177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:00.944612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:00.956741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 463/920] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:00.962817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:00.980071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:01.071261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2952d-496e-45a4-9270-001bca031198 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:01.110093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:01.110792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:01.111071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:01.111343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:01.118347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:01.119790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:14:01.131325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:01.137324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:01.138479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 458/921] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:14:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:01.140413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:01.224550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:01.248256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c127a-c369-419b-ad80-d756881c9d6a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:01.291190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949380a-5159-4ee2-a506-2c50c27246d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:01.292215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949380a-5159-4ee2-a506-2c50c27246d7 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:01.292697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949380a-5159-4ee2-a506-2c50c27246d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:01.293069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949380a-5159-4ee2-a506-2c50c27246d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:01.293807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949380a-5159-4ee2-a506-2c50c27246d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:01.294170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949380a-5159-4ee2-a506-2c50c27246d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:01.295711 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949380a-5159-4ee2-a506-2c50c27246d7 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:01.310267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 464/922] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:14:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:01.317131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:01.400954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:01.426364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c4cd1-d4b2-4238-8656-ada5fb68f7a0 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:01.465434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58c612-d371-40b3-9a22-11c182795449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:01.466115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c612-d371-40b3-9a22-11c182795449 None admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:01.466387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c612-d371-40b3-9a22-11c182795449 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:01.466650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c612-d371-40b3-9a22-11c182795449 None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/a578acf1e0cc4505a03c1b587b381b04/roles/eb326c809f1543ce8be84f07a0a1cdab` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:01.484011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58c612-d371-40b3-9a22-11c182795449 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=a578acf1e0cc4505a03c1b587b381b04, role_id=eb326c809f1543ce8be84f07a0a1cdab)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:01.486160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58c612-d371-40b3-9a22-11c182795449 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:01.501342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 459/923] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:14:01 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/a578acf1e0cc4505a03c1b587b381b04/roles/eb326c809f1543ce8be84f07a0a1cdab => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:14:03.621821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02723a-1539-4181-8a1e-fda0e92f51f8 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:03.622279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02723a-1539-4181-8a1e-fda0e92f51f8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:03.622367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02723a-1539-4181-8a1e-fda0e92f51f8 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:03.623695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 465/924] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:14:03.629635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e3198-0232-4d3b-a5f5-5e5115b5202a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:03.630593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e3198-0232-4d3b-a5f5-5e5115b5202a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:03.630981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e3198-0232-4d3b-a5f5-5e5115b5202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:03.649892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4e5e3198-0232-4d3b-a5f5-5e5115b5202a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:14:03.651132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4e5e3198-0232-4d3b-a5f5-5e5115b5202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:14:03.720334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 460/925] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:14:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:14:03.993606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02723a-1539-4181-8a1e-fda0e92f51f8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:04.096898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02723a-1539-4181-8a1e-fda0e92f51f8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:04.131499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:04.132117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:04.132429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:04.132700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:04.139542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:04.143189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:14:04.148540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:04.149970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:04.151002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 466/926] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Oct 27 13:14:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:04.155905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e3198-0232-4d3b-a5f5-5e5115b5202a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:04.221252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:04.270980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e3198-0232-4d3b-a5f5-5e5115b5202a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:04.307243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:04.308050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:04.308391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:04.308804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:04.309604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:04.309948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:04.311718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:04.320948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 461/927] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Oct 27 13:14:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:04.328282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:04.441018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944af2c-f06d-4060-ad30-88b3907d787d None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:04.475505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:04.476119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:04.476442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:04.476683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:04.487308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:04.488778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:04.503823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:04.504998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:04.506001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:04.506001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:04.506001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:04.506001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 467/928] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:04.511357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:04.619714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab345df-9cf3-4941-b737-794db57925b6 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:04.636100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:04.657835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:04.658444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:04.658722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:04.658993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:04.659625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:04.659926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:04.661319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:04.674964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 462/929] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:04.680959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:04.735829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:04.790867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1e5c8-a599-426e-9d1a-403a44d70262 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:04.827253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:04.828462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:04.828767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:04.829072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] PATH_INFO: `/v3/users/placement` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:04.837168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:04.838602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:14:04.850801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:04.852211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:04.853479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 468/930] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:04.858063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:04.967275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3355fd7-611a-4803-9062-babcedbdea60 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:05.004655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:05.005265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:05.005557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:05.005926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:05.006958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:05.007064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:05.008641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:05.029242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 463/931] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Oct 27 13:14:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:05.035880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:05.146804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27546214-03fd-4bdd-9ce8-56b0ad9118de None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:05.157852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:05.181821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:05.182532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:05.182778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:05.183084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:05.259397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:05.260866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:05.373870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 469/932] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:14:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:14:05.381100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:05.420557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:05.496403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55163b92-b927-4d7e-93a3-7e0e53da460a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:05.534437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:05.534930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:05.535223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:05.535493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:05.551257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:05.552721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:05.569748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:05.570923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:05.572010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:05.572010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:05.572010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:05.576859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:05.579856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 464/933] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:05.590154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:05.689477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2eb3a-e8f1-4fea-a5e8-2fe0823c0fd8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:05.723654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:05.724274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:05.724555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:05.724824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:05.725450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:05.725745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:05.727098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:05.738583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 470/934] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:05.744309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:05.843102 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:05.860545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76927-eae6-42f0-beda-7cabda6c64fb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:05.897343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:05.897959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:05.898235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:05.898501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:05.905304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:05.907033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:14:05.920097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:05.921142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:05.922659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 465/935] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:14:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:05.928593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:06.009121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:06.043853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aae71-38d6-4b9f-8a81-d978a667b4f8 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:06.081947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b4772-eeb6-45b2-b3db-a61b4e6afc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:06.082539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4772-eeb6-45b2-b3db-a61b4e6afc86 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:06.082819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4772-eeb6-45b2-b3db-a61b4e6afc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:06.083090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4772-eeb6-45b2-b3db-a61b4e6afc86 None admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:06.083773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4772-eeb6-45b2-b3db-a61b4e6afc86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:06.084108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4772-eeb6-45b2-b3db-a61b4e6afc86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:06.085640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4772-eeb6-45b2-b3db-a61b4e6afc86 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:06.098630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 471/936] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:14:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:06.104760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:06.208835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21360d5-b644-451f-bb39-7b6f3d95b29b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:06.244815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175aa7c-7ec3-4206-85ca-cc3fbb454101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:06.245424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175aa7c-7ec3-4206-85ca-cc3fbb454101 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:06.245704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175aa7c-7ec3-4206-85ca-cc3fbb454101 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:06.245974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175aa7c-7ec3-4206-85ca-cc3fbb454101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:06.246686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175aa7c-7ec3-4206-85ca-cc3fbb454101 None admin] RBAC: Adding query filter params (role.id=eb326c809f1543ce8be84f07a0a1cdab, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=a578acf1e0cc4505a03c1b587b381b04) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:06.246991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175aa7c-7ec3-4206-85ca-cc3fbb454101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:06.248425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175aa7c-7ec3-4206-85ca-cc3fbb454101 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:06.254795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 466/937] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:14:06 2022] GET /identity/v3/role_assignments?user.id=a578acf1e0cc4505a03c1b587b381b04&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=eb326c809f1543ce8be84f07a0a1cdab => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:08.411870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4ccdb-3ea6-4def-a14d-2064f8e08282 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:08.412429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4ccdb-3ea6-4def-a14d-2064f8e08282 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:08.412429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4ccdb-3ea6-4def-a14d-2064f8e08282 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:08.413743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 472/938] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:14:08.419393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72ceb9-44e2-459d-9d7d-bfe8a1052400 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:08.419909 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72ceb9-44e2-459d-9d7d-bfe8a1052400 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:08.420011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72ceb9-44e2-459d-9d7d-bfe8a1052400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:08.439899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bf72ceb9-44e2-459d-9d7d-bfe8a1052400 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:14:08.441216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bf72ceb9-44e2-459d-9d7d-bfe8a1052400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:14:08.445513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:08.518406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 467/939] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:14:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:14:08.787323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4ccdb-3ea6-4def-a14d-2064f8e08282 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:08.897182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4ccdb-3ea6-4def-a14d-2064f8e08282 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:08.926095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:08.932859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:08.933484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:08.933759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:08.934030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:08.944613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:14:08.949267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:08.950192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:14:08.951010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:08.956011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 473/940] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:08.956942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72ceb9-44e2-459d-9d7d-bfe8a1052400 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:09.057272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72ceb9-44e2-459d-9d7d-bfe8a1052400 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:09.097329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:09.098092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:09.098451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:09.098825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:09.099629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:09.100046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:09.101839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:09.111501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 468/941] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:09.117459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:09.224714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef920-f5b0-4377-ab4e-bbc3f17632d2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:09.264239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:09.264965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:09.265816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:09.266082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:09.277244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:09.278691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:09.297818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:09.299077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:09.300201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:09.300201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:09.300201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:09.307456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 474/942] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:09.308376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:09.311302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:09.416806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5ea4-7ed2-453f-98b0-35f727b4fe2f None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:09.455576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:09.456293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:09.456552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:09.456845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:09.457473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:09.457770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:09.459118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:09.471293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 469/943] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:09.477310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:09.524842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:09.581552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41e27d-a4fc-4d4d-b26d-9155a9e83a8e None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:09.616616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:09.617399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:09.617606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:09.617879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:09.626326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:09.627856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:14:09.639708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:09.641322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:09.642444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 475/944] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:09.646716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:09.709892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:09.749211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13269e7-4e60-4b1a-a3ef-8e36d88ac4eb None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:09.786934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:09.787669 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:09.787947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:09.788214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:09.788957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:09.790085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:09.791787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:09.812307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 470/945] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Oct 27 13:14:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:09.818689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:09.932540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c629-a4f9-47f1-99ab-3e61f22f95dd None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:09.934159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:09.973513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:09.974152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:09.974473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:09.974746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:09.989618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:09.991096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:10.005301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:10.006334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:10.012056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:10.012056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:10.012056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:10.012056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 476/946] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:10.013397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:10.120441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb951f-6762-451a-9d27-e1dc68315f82 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:10.145215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:10.160823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:10.161434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:10.161709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:10.161990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:10.162593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:10.163029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:10.164248 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:10.175749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 471/947] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:10.181983 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:10.289266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecd036-dd86-475e-8f0e-cb367c61c719 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:10.326390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:10.327070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:10.327365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:10.327655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:10.334921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:10.336372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:14:10.347823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:10.348843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:10.349743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 477/948] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:14:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:10.355072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:10.462059 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2398d1-c962-4061-bf2b-df4c32fbe57b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:10.497268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9dc57-e76a-4ac2-86cb-6b495936690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:10.497869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9dc57-e76a-4ac2-86cb-6b495936690f None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:10.498147 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9dc57-e76a-4ac2-86cb-6b495936690f None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:10.498410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9dc57-e76a-4ac2-86cb-6b495936690f None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:10.499087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9dc57-e76a-4ac2-86cb-6b495936690f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:10.499384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9dc57-e76a-4ac2-86cb-6b495936690f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:10.500817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9dc57-e76a-4ac2-86cb-6b495936690f None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:10.512960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 472/949] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:10.519323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:10.536025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:10.625806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef7e3-d5f8-4aef-85eb-772dd10ac998 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:10.661829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a459a-9c94-464d-8ad4-fddabd24512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:10.662498 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a459a-9c94-464d-8ad4-fddabd24512d None admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:10.662769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a459a-9c94-464d-8ad4-fddabd24512d None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:10.663041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a459a-9c94-464d-8ad4-fddabd24512d None admin] PATH_INFO: `/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/a578acf1e0cc4505a03c1b587b381b04/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:10.665009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a459a-9c94-464d-8ad4-fddabd24512d None admin] RBAC: Authorizing `identity:create_grant(project_id=4aba5f43c2f94c27a9d4152aa186289e, user_id=a578acf1e0cc4505a03c1b587b381b04, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:10.666436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a459a-9c94-464d-8ad4-fddabd24512d None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:10.680557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 478/950] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Oct 27 13:14:10 2022] PUT /identity/v3/projects/4aba5f43c2f94c27a9d4152aa186289e/users/a578acf1e0cc4505a03c1b587b381b04/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:14:13.004028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6afb-f160-49f6-87bf-776cbafbdbdb None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:13.004585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6afb-f160-49f6-87bf-776cbafbdbdb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:13.004585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6afb-f160-49f6-87bf-776cbafbdbdb None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:13.006029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 473/951] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:14:13.012257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6cfff-9567-4ad7-9e1c-5f37ce25f91a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:13.012668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6cfff-9567-4ad7-9e1c-5f37ce25f91a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:13.012888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6cfff-9567-4ad7-9e1c-5f37ce25f91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:13.033469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-daa6cfff-9567-4ad7-9e1c-5f37ce25f91a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:14:13.034822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-daa6cfff-9567-4ad7-9e1c-5f37ce25f91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:14:13.107326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 479/952] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:14:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:14:13.400495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b6afb-f160-49f6-87bf-776cbafbdbdb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:13.445971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:13.504214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b6afb-f160-49f6-87bf-776cbafbdbdb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:13.540298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:13.540932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:13.541210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:13.541483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:13.554058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 27 13:14:13.559215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:13.560402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:13.561221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 474/953] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Oct 27 13:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:13.566964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6cfff-9567-4ad7-9e1c-5f37ce25f91a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:13.570813 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:13.683328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6cfff-9567-4ad7-9e1c-5f37ce25f91a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:13.718620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:13.719222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:13.719495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:13.719761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:13.720506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:13.720799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:13.722145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:13.730446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 480/954] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Oct 27 13:14:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:13.737111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:13.841176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:13.844455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd30e7-0d29-452d-967d-cfc3e4366880 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:13.884203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:13.884921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:13.885207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:13.885486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:13.896672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:13.898138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:13.914015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:13.914990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:13.915869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:13.915869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:13.915869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:13.915869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 475/955] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:13.922375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:13.990626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:14.029218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91551da3-b212-4549-ad91-1e78df84918a None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:14.065277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:14.065939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:14.066268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:14.066545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:14.067262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:14.067574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:14.069095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:14.081330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 481/956] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:14.087308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:14.195104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36354f7-b285-4b51-b756-1cbfa5a4355b None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:14.230842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:14.231473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:14.231764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:14.232032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:14.239714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:14.241240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 27 13:14:14.253623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:14.254939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:14.256257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 476/957] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:14.260837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:14.368892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45730d34-e99a-4f82-9537-b71c43b40244 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:14.399576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:14.408539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:14.409138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:14.409412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:14.409676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:14.410378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:14.410675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:14.412107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:14.429409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 482/958] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Oct 27 13:14:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:14.435159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:14.452027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:14.543710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60a64a-786a-4a1f-80fd-6dcf983dae7d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:14.579614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:14.580277 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:14.580603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:14.580877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:14.654812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:14.656304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:14.669977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:14.761356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 477/959] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Oct 27 13:14:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 27 13:14:14.768533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:14.875750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8ce60-d09a-4b6f-ba44-bda146bfb2f2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:14.912503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:14.913230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:14.913602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:14.913883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:14.926240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:14.927765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 27 13:14:14.946773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:14.947933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 27 13:14:14.949004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 27 13:14:14.949004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 27 13:14:14.949004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:14.949004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 483/960] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:14.953766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:15.060845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a755b4a-ee80-4940-9421-758e3d03221a None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:15.074781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:15.097204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:15.097924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:15.098090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:15.098359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:15.099013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:15.099311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:15.100743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:15.114018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 478/961] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Oct 27 13:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:15.119787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:15.157712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:15.223398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf9ac6-2d80-4d91-ba89-795a501493f2 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:15.258811 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:15.259439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:15.259756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:15.259986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:15.268042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:15.269528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 27 13:14:15.282788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:15.283930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 27 13:14:15.285052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: ERROR keystone.server.flask.application Oct 27 13:14:15.288805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 484/962] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Oct 27 13:14:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:15.290338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:15.402384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafbd55-c3a9-41c0-b742-a955d9507944 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:15.437455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6676a-b50f-4ffa-ba01-c20dd5775c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:15.437985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6676a-b50f-4ffa-ba01-c20dd5775c21 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:15.438262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6676a-b50f-4ffa-ba01-c20dd5775c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:15.438533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6676a-b50f-4ffa-ba01-c20dd5775c21 None admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:15.439217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6676a-b50f-4ffa-ba01-c20dd5775c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:15.439524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6676a-b50f-4ffa-ba01-c20dd5775c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:15.440998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6676a-b50f-4ffa-ba01-c20dd5775c21 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:15.453568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 479/963] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Oct 27 13:14:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:15.459829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:15.556433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:15.562871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2edb08-463c-46af-b2bb-44487eb61e6c None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:15.597363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a8381-0832-4629-b86a-4341fd1f86e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:15.597976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8381-0832-4629-b86a-4341fd1f86e8 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:15.598249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8381-0832-4629-b86a-4341fd1f86e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:15.598520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a8381-0832-4629-b86a-4341fd1f86e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:15.599238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a8381-0832-4629-b86a-4341fd1f86e8 None admin] RBAC: Adding query filter params (role.id=33f912ad05b44e36a696ef6aa86733be, scope.project.id=4aba5f43c2f94c27a9d4152aa186289e, user.id=a578acf1e0cc4505a03c1b587b381b04) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:15.599543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a8381-0832-4629-b86a-4341fd1f86e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:15.600993 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a8381-0832-4629-b86a-4341fd1f86e8 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:15.607082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 485/964] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Oct 27 13:14:15 2022] GET /identity/v3/role_assignments?user.id=a578acf1e0cc4505a03c1b587b381b04&scope.project.id=4aba5f43c2f94c27a9d4152aa186289e&role.id=33f912ad05b44e36a696ef6aa86733be => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:17.743166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b99f6-bfb4-40e8-b11b-e6199670e1f6 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:17.743671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b99f6-bfb4-40e8-b11b-e6199670e1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:17.743671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b99f6-bfb4-40e8-b11b-e6199670e1f6 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:17.745037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 480/965] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:14:17.750230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5f3cc-e48c-4981-8cc1-5a8e640abe17 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:17.750605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5f3cc-e48c-4981-8cc1-5a8e640abe17 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:17.750765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5f3cc-e48c-4981-8cc1-5a8e640abe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:17.769040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-adf5f3cc-e48c-4981-8cc1-5a8e640abe17 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:14:17.770193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-adf5f3cc-e48c-4981-8cc1-5a8e640abe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:14:17.832662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 486/966] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:14:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:14:18.100194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b99f6-bfb4-40e8-b11b-e6199670e1f6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:18.130527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:18.198230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b99f6-bfb4-40e8-b11b-e6199670e1f6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:18.230486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:18.231094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:18.231367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:18.231633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:18.232193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:18.233625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:18.242082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:18.243240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:18.244265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 481/967] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:14:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:18.249560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5f3cc-e48c-4981-8cc1-5a8e640abe17 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:18.343884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:18.353852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5f3cc-e48c-4981-8cc1-5a8e640abe17 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:18.385453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12f048-0d77-420f-8092-ac92f17bf936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:18.386042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12f048-0d77-420f-8092-ac92f17bf936 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:18.386314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12f048-0d77-420f-8092-ac92f17bf936 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:18.386577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12f048-0d77-420f-8092-ac92f17bf936 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:18.387181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12f048-0d77-420f-8092-ac92f17bf936 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:18.387470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12f048-0d77-420f-8092-ac92f17bf936 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:18.389083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12f048-0d77-420f-8092-ac92f17bf936 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:18.393644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 487/968] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Oct 27 13:14:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:18.399373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:18.490301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:18.496435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2805b-39b0-466b-888c-9ca71d5cd279 None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:18.528860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e165b6-48dd-4759-81d8-ae8c29cd23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:18.529446 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e165b6-48dd-4759-81d8-ae8c29cd23ae None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:18.529720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e165b6-48dd-4759-81d8-ae8c29cd23ae None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:18.530027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e165b6-48dd-4759-81d8-ae8c29cd23ae None admin] PATH_INFO: `/v3/services` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:18.530638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e165b6-48dd-4759-81d8-ae8c29cd23ae None admin] RBAC: Adding query filter params (type=placement) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:18.530934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e165b6-48dd-4759-81d8-ae8c29cd23ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:18.532285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e165b6-48dd-4759-81d8-ae8c29cd23ae None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:18.536500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 482/969] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Oct 27 13:14:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:20.662919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe9e51-67bc-4697-aa70-9b536dd5694d None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:20.663408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe9e51-67bc-4697-aa70-9b536dd5694d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:20.663499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe9e51-67bc-4697-aa70-9b536dd5694d None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:20.664792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 488/970] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:14:20.670029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14909478-f7c7-49be-b08f-c47e189c4aae None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:20.670952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14909478-f7c7-49be-b08f-c47e189c4aae None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:20.671095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14909478-f7c7-49be-b08f-c47e189c4aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:20.689540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-14909478-f7c7-49be-b08f-c47e189c4aae None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:14:20.690716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-14909478-f7c7-49be-b08f-c47e189c4aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:14:20.755187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 483/971] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:14:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:14:21.019342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe9e51-67bc-4697-aa70-9b536dd5694d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:21.116026 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe9e51-67bc-4697-aa70-9b536dd5694d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:21.147527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebfb06-c44b-43a9-83a3-66d94ca91108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:21.148112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebfb06-c44b-43a9-83a3-66d94ca91108 None admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:21.148410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebfb06-c44b-43a9-83a3-66d94ca91108 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:21.148674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebfb06-c44b-43a9-83a3-66d94ca91108 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:21.149298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebfb06-c44b-43a9-83a3-66d94ca91108 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:21.150621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebfb06-c44b-43a9-83a3-66d94ca91108 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:21.164776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 489/972] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Oct 27 13:14:21 2022] POST /identity/v3/services => generated 246 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:14:23.307027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951cd23-965f-40ca-9cdb-0e39a580b149 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:23.307703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951cd23-965f-40ca-9cdb-0e39a580b149 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:23.307703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951cd23-965f-40ca-9cdb-0e39a580b149 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:23.309026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 484/973] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:14:23.314919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86036-686f-47db-ab0f-05a22ee6cd86 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:23.315360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86036-686f-47db-ab0f-05a22ee6cd86 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:23.315468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86036-686f-47db-ab0f-05a22ee6cd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:23.334082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-dfe86036-686f-47db-ab0f-05a22ee6cd86 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:14:23.335238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-dfe86036-686f-47db-ab0f-05a22ee6cd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:14:23.344004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:23.405917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 490/974] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:14:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 27 13:14:23.668183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951cd23-965f-40ca-9cdb-0e39a580b149 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:23.756041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:23.766831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951cd23-965f-40ca-9cdb-0e39a580b149 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:23.799101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:23.799710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:23.799983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:23.800273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:23.800844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:23.802184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:23.810599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:23.812409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:23.813495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 485/975] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:14:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:23.818308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86036-686f-47db-ab0f-05a22ee6cd86 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:23.910760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86036-686f-47db-ab0f-05a22ee6cd86 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:23.943263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:23.943851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:23.944125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:23.944421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:23.945030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:23.945323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:23.946653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:23.951520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 491/976] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Oct 27 13:14:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:23.957190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:24.056822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e6e3-ba39-4619-8503-af977935828d None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:24.088482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c079fda-b357-453d-a872-d3a3a6ed5513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:24.089068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c079fda-b357-453d-a872-d3a3a6ed5513 None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:24.089340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c079fda-b357-453d-a872-d3a3a6ed5513 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:24.089599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c079fda-b357-453d-a872-d3a3a6ed5513 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:24.090271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c079fda-b357-453d-a872-d3a3a6ed5513 None admin] RBAC: Adding query filter params (interface=public, service_id=26ec15678b974517aeeb6f39bd19ec69, region_id=RegionOne) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:24.090569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c079fda-b357-453d-a872-d3a3a6ed5513 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:24.091894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c079fda-b357-453d-a872-d3a3a6ed5513 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:24.097204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 486/977] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Oct 27 13:14:23 2022] GET /identity/v3/endpoints?service_id=26ec15678b974517aeeb6f39bd19ec69&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:14:24.102820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:24.106451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:24.198166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81ae5-1fac-443c-81e9-2d3b360d0ca9 None admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:24.229665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4f5df-afdf-4625-99ee-7200a8a36300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:24.230255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f5df-afdf-4625-99ee-7200a8a36300 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:24.230528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f5df-afdf-4625-99ee-7200a8a36300 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:24.230793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f5df-afdf-4625-99ee-7200a8a36300 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:24.231375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4f5df-afdf-4625-99ee-7200a8a36300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:24.233327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4f5df-afdf-4625-99ee-7200a8a36300 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:24.239747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 492/978] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Oct 27 13:14:24 2022] GET /identity/v3/services => generated 2191 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:14:26.386138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54251b26-8507-44ed-b631-7533be9d648d None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:26.386737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54251b26-8507-44ed-b631-7533be9d648d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:26.386844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54251b26-8507-44ed-b631-7533be9d648d None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:26.388513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 487/979] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:14:26.394500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8fef3-9d86-4b93-a584-27b0641e4952 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:26.395373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8fef3-9d86-4b93-a584-27b0641e4952 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:26.395373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8fef3-9d86-4b93-a584-27b0641e4952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:26.414012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-69a8fef3-9d86-4b93-a584-27b0641e4952 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:14:26.415166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-69a8fef3-9d86-4b93-a584-27b0641e4952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:14:26.473414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:26.478765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 493/980] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:14:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 27 13:14:26.747367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54251b26-8507-44ed-b631-7533be9d648d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:26.767260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:26.844883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54251b26-8507-44ed-b631-7533be9d648d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:26.876757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:26.877359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:26.877630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:26.877894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:26.878449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:26.879775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 27 13:14:26.888340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 27 13:14:26.889482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.get_or_create( Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application with Lock( Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self._enter() Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application return self.creator() Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application created_value = creator( Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: ERROR keystone.server.flask.application Oct 27 13:14:26.890630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 488/981] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Oct 27 13:14:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 27 13:14:26.895299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8fef3-9d86-4b93-a584-27b0641e4952 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:26.989279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8fef3-9d86-4b93-a584-27b0641e4952 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:27.021129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4c489-87cf-49aa-a978-18c54a2e1f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:27.021661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4c489-87cf-49aa-a978-18c54a2e1f26 None admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:27.021936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4c489-87cf-49aa-a978-18c54a2e1f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:27.022203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4c489-87cf-49aa-a978-18c54a2e1f26 None admin] PATH_INFO: `/v3/services` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:27.022808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4c489-87cf-49aa-a978-18c54a2e1f26 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:14:27.023105 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4c489-87cf-49aa-a978-18c54a2e1f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:27.024465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4c489-87cf-49aa-a978-18c54a2e1f26 None admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:27.029817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 494/982] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Oct 27 13:14:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:14:27.035215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:14:27.113366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:27.131011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e6d7-741c-4ba5-b3c6-75788017a54c None admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:14:27.163642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe5f75-4f64-487d-a45b-9d0edafe7053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:14:27.164245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe5f75-4f64-487d-a45b-9d0edafe7053 None admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:14:27.164522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe5f75-4f64-487d-a45b-9d0edafe7053 None admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:14:27.164784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe5f75-4f64-487d-a45b-9d0edafe7053 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:14:27.165436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe5f75-4f64-487d-a45b-9d0edafe7053 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:14:27.166762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe5f75-4f64-487d-a45b-9d0edafe7053 None admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:14:27.185428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 489/983] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:14:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:14:37.027499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=69 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:14:37.181847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:30.322388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1579-6b5b-4420-b48c-b8ae043a7aea None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:30.323785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1579-6b5b-4420-b48c-b8ae043a7aea None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:30.324466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1579-6b5b-4420-b48c-b8ae043a7aea None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:30.330064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 495/984] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:30.331987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37abd3-831c-4da0-b382-5bd89cf3dadf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:30.332459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37abd3-831c-4da0-b382-5bd89cf3dadf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:30.332540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37abd3-831c-4da0-b382-5bd89cf3dadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:30.450434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bf37abd3-831c-4da0-b382-5bd89cf3dadf None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:30.455739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bf37abd3-831c-4da0-b382-5bd89cf3dadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:30.720680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 490/985] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:30.994297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a47693-e9a9-4508-9495-13a6870e5e85 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:30.994986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a47693-e9a9-4508-9495-13a6870e5e85 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:30.995459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a47693-e9a9-4508-9495-13a6870e5e85 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:30.997277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 496/986] 10.222.0.48 () {54 vars in 965 bytes} [Thu Oct 27 13:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:31.004830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92908a-0757-4adc-accb-02b040004c11 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:31.005430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92908a-0757-4adc-accb-02b040004c11 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:31.005906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92908a-0757-4adc-accb-02b040004c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:31.090585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bf92908a-0757-4adc-accb-02b040004c11 None None] MFA Rules not processed for user `b7cba7b6049e41488f5b2f7c61a0500a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:31.091796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bf92908a-0757-4adc-accb-02b040004c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:31.117749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:31.185211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 491/987] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Oct 27 13:15:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:31.195155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a47693-e9a9-4508-9495-13a6870e5e85 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:31.197178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-17a47693-e9a9-4508-9495-13a6870e5e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:31.328736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a47693-e9a9-4508-9495-13a6870e5e85 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:31.414074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2cdf1-76d0-4082-872c-7a18881cd884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:31.414944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2cdf1-76d0-4082-872c-7a18881cd884 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:31.415373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2cdf1-76d0-4082-872c-7a18881cd884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:31.415791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2cdf1-76d0-4082-872c-7a18881cd884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:31.417763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a7c2cdf1-76d0-4082-872c-7a18881cd884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:31.479919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:31.549861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2cdf1-76d0-4082-872c-7a18881cd884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:31.551357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2cdf1-76d0-4082-872c-7a18881cd884 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:31.575923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 497/988] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:15:34.072904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb1cce-7f3c-4741-9bef-ee63549d2f01 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:34.073449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb1cce-7f3c-4741-9bef-ee63549d2f01 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:34.073449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb1cce-7f3c-4741-9bef-ee63549d2f01 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:34.074779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 492/989] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:34.079994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f241e5-ddfa-4638-ae0c-24c508246061 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:34.080458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f241e5-ddfa-4638-ae0c-24c508246061 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:34.080596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f241e5-ddfa-4638-ae0c-24c508246061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:34.163654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-94f241e5-ddfa-4638-ae0c-24c508246061 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:34.164809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-94f241e5-ddfa-4638-ae0c-24c508246061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:34.260724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 498/990] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:34.547372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb1cce-7f3c-4741-9bef-ee63549d2f01 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:34.548937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bdbb1cce-7f3c-4741-9bef-ee63549d2f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:34.651898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb1cce-7f3c-4741-9bef-ee63549d2f01 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:34.721082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:34.723484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0e7ee-9cb2-40ba-9115-80d6304041c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:34.724344 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0e7ee-9cb2-40ba-9115-80d6304041c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:34.724674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0e7ee-9cb2-40ba-9115-80d6304041c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:34.725076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0e7ee-9cb2-40ba-9115-80d6304041c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:34.726850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bab0e7ee-9cb2-40ba-9115-80d6304041c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:34.833289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0e7ee-9cb2-40ba-9115-80d6304041c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:34.834647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0e7ee-9cb2-40ba-9115-80d6304041c8 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:34.852587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 493/991] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:15:37.290863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d7f97-fea0-42e4-9056-cf99e62a16b1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:37.291388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d7f97-fea0-42e4-9056-cf99e62a16b1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:37.291388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d7f97-fea0-42e4-9056-cf99e62a16b1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:37.292686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 499/992] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:37.297741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9bcfc-5761-4418-b304-1ac8ddb19f7b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:37.298158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9bcfc-5761-4418-b304-1ac8ddb19f7b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:37.298287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9bcfc-5761-4418-b304-1ac8ddb19f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:37.363322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a5c9bcfc-5761-4418-b304-1ac8ddb19f7b None None] MFA Rules not processed for user `b3d5364c4bfc467ba54c1022c639c48c`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:37.364522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a5c9bcfc-5761-4418-b304-1ac8ddb19f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:37.415593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:37.459581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 494/993] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:15:37.739543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d7f97-fea0-42e4-9056-cf99e62a16b1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:37.741118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d62d7f97-fea0-42e4-9056-cf99e62a16b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:37.752100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:37.842999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d7f97-fea0-42e4-9056-cf99e62a16b1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:37.913173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51209890-ce5c-4b0e-9f0a-db07f0c76592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:37.913700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209890-ce5c-4b0e-9f0a-db07f0c76592 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:37.913971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209890-ce5c-4b0e-9f0a-db07f0c76592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:37.914235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209890-ce5c-4b0e-9f0a-db07f0c76592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:37.916395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-51209890-ce5c-4b0e-9f0a-db07f0c76592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:37.975833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:38.021709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51209890-ce5c-4b0e-9f0a-db07f0c76592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:38.023115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51209890-ce5c-4b0e-9f0a-db07f0c76592 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:38.041186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 500/994] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:15:40.744388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4fab6-6627-4dbb-81f3-ba820ca7ca39 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:40.744930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4fab6-6627-4dbb-81f3-ba820ca7ca39 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:40.744930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4fab6-6627-4dbb-81f3-ba820ca7ca39 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:40.746349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 495/995] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:40.751599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735775e-e57d-4952-8cdd-163f79d6bbd9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:40.751995 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735775e-e57d-4952-8cdd-163f79d6bbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:40.752133 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735775e-e57d-4952-8cdd-163f79d6bbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:40.816573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b735775e-e57d-4952-8cdd-163f79d6bbd9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:40.817745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b735775e-e57d-4952-8cdd-163f79d6bbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:40.911472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 501/996] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:41.193428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4fab6-6627-4dbb-81f3-ba820ca7ca39 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:41.194954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cac4fab6-6627-4dbb-81f3-ba820ca7ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:41.298451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4fab6-6627-4dbb-81f3-ba820ca7ca39 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:41.369938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572b231-f6fd-4032-8763-49ef8cd44b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:41.370536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572b231-f6fd-4032-8763-49ef8cd44b0a service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:41.370808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572b231-f6fd-4032-8763-49ef8cd44b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:41.371074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572b231-f6fd-4032-8763-49ef8cd44b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:41.372692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7572b231-f6fd-4032-8763-49ef8cd44b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:41.379366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:41.479427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572b231-f6fd-4032-8763-49ef8cd44b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:41.480801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572b231-f6fd-4032-8763-49ef8cd44b0a service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:41.498083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 496/997] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:15:44.258412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fb9c8-1dba-4111-a0fa-e07ab46c02bc None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:44.259034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fb9c8-1dba-4111-a0fa-e07ab46c02bc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:44.259034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fb9c8-1dba-4111-a0fa-e07ab46c02bc None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:44.260632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 502/998] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:44.266311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e7751-d3ea-4401-816e-aa8bc1605b58 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:44.266736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e7751-d3ea-4401-816e-aa8bc1605b58 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:44.266853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e7751-d3ea-4401-816e-aa8bc1605b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:44.341059 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e68e7751-d3ea-4401-816e-aa8bc1605b58 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:44.342281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e68e7751-d3ea-4401-816e-aa8bc1605b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:44.404584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:44.446443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 497/999] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:44.728349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fb9c8-1dba-4111-a0fa-e07ab46c02bc None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:44.729866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e01fb9c8-1dba-4111-a0fa-e07ab46c02bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:44.749423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:44.831661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fb9c8-1dba-4111-a0fa-e07ab46c02bc None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:44.903104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaae24-7cc1-4078-8dc3-6fae7a32447a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:44.903703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaae24-7cc1-4078-8dc3-6fae7a32447a service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:44.903976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaae24-7cc1-4078-8dc3-6fae7a32447a service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:44.904251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaae24-7cc1-4078-8dc3-6fae7a32447a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:44.905907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-26aaae24-7cc1-4078-8dc3-6fae7a32447a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:44.975344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:45.012839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aaae24-7cc1-4078-8dc3-6fae7a32447a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:45.014258 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aaae24-7cc1-4078-8dc3-6fae7a32447a service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:45.031641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 503/1000] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:15:47.238988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37507f39-e92e-4819-8c57-42161ee0cfbc None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:47.239535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37507f39-e92e-4819-8c57-42161ee0cfbc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:47.239535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37507f39-e92e-4819-8c57-42161ee0cfbc None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:47.240876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 498/1001] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:47.246176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f2137-0dd6-4a7d-bb5b-922bc91e39d5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:47.246611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f2137-0dd6-4a7d-bb5b-922bc91e39d5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:47.246701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f2137-0dd6-4a7d-bb5b-922bc91e39d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:47.311650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b53f2137-0dd6-4a7d-bb5b-922bc91e39d5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:47.312771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b53f2137-0dd6-4a7d-bb5b-922bc91e39d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:47.420128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 504/1002] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:47.705227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37507f39-e92e-4819-8c57-42161ee0cfbc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:47.706813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-37507f39-e92e-4819-8c57-42161ee0cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:47.810485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37507f39-e92e-4819-8c57-42161ee0cfbc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:47.882604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843354b5-2d76-41bc-949d-c5cf4c77d4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:47.883124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843354b5-2d76-41bc-949d-c5cf4c77d4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:47.883478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843354b5-2d76-41bc-949d-c5cf4c77d4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:47.883747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843354b5-2d76-41bc-949d-c5cf4c77d4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:47.885625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-843354b5-2d76-41bc-949d-c5cf4c77d4e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:47.902575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:47.995044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843354b5-2d76-41bc-949d-c5cf4c77d4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:47.996497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843354b5-2d76-41bc-949d-c5cf4c77d4e5 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:48.014156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 499/1003] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:15:50.215328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14df42-ce60-4bd6-a8f4-4b9f3fff565e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:50.215845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14df42-ce60-4bd6-a8f4-4b9f3fff565e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:50.215845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14df42-ce60-4bd6-a8f4-4b9f3fff565e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:50.217454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 505/1004] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:50.222515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9389a-fe8e-4186-90d7-f147d3bc3848 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:50.222938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9389a-fe8e-4186-90d7-f147d3bc3848 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:50.223051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9389a-fe8e-4186-90d7-f147d3bc3848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:50.293065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f1f9389a-fe8e-4186-90d7-f147d3bc3848 None None] MFA Rules not processed for user `b3d5364c4bfc467ba54c1022c639c48c`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:50.294370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f1f9389a-fe8e-4186-90d7-f147d3bc3848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:50.365651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:50.400197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 500/1005] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:50.701046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6950e-ce80-4875-96ad-113f6a922dca None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:50.701445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6950e-ce80-4875-96ad-113f6a922dca None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:50.701780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6950e-ce80-4875-96ad-113f6a922dca None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:50.703203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 506/1006] 10.222.0.48 () {54 vars in 965 bytes} [Thu Oct 27 13:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:50.709567 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aecd01-0faa-466a-a99b-9f0263a55635 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:50.709775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aecd01-0faa-466a-a99b-9f0263a55635 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:50.710051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aecd01-0faa-466a-a99b-9f0263a55635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:50.787478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-16aecd01-0faa-466a-a99b-9f0263a55635 None None] MFA Rules not processed for user `b7cba7b6049e41488f5b2f7c61a0500a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:50.788633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-16aecd01-0faa-466a-a99b-9f0263a55635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:50.864866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 501/1007] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Oct 27 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:50.873902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6950e-ce80-4875-96ad-113f6a922dca None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:50.875411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-84c6950e-ce80-4875-96ad-113f6a922dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:50.907694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:50.979334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6950e-ce80-4875-96ad-113f6a922dca None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:51.051401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902f52a-c4d3-42c8-9dcb-6e2a5beddc84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:51.051916 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f52a-c4d3-42c8-9dcb-6e2a5beddc84 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:51.052185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f52a-c4d3-42c8-9dcb-6e2a5beddc84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:51.052478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f52a-c4d3-42c8-9dcb-6e2a5beddc84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:51.054034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c902f52a-c4d3-42c8-9dcb-6e2a5beddc84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:51.128388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:51.161707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902f52a-c4d3-42c8-9dcb-6e2a5beddc84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:51.163061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902f52a-c4d3-42c8-9dcb-6e2a5beddc84 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:51.182031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 507/1008] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:15:53.903954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6bdcf-fd6b-4ea8-bd75-f5864af78c19 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:53.904496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6bdcf-fd6b-4ea8-bd75-f5864af78c19 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:53.904496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6bdcf-fd6b-4ea8-bd75-f5864af78c19 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:53.905808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 502/1009] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:53.910847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bece151-4e25-4b20-99a6-9f776d4eed30 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:53.911231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bece151-4e25-4b20-99a6-9f776d4eed30 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:53.911386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bece151-4e25-4b20-99a6-9f776d4eed30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:53.975178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5bece151-4e25-4b20-99a6-9f776d4eed30 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:53.976371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5bece151-4e25-4b20-99a6-9f776d4eed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:54.069997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 508/1010] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:54.347232 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6bdcf-fd6b-4ea8-bd75-f5864af78c19 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:54.348778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b3a6bdcf-fd6b-4ea8-bd75-f5864af78c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:54.418016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:54.450821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6bdcf-fd6b-4ea8-bd75-f5864af78c19 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:54.524355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b2ab5-399e-4c33-9a59-9689dcb79b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:54.524948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b2ab5-399e-4c33-9a59-9689dcb79b3f service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:54.525218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b2ab5-399e-4c33-9a59-9689dcb79b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:54.525479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b2ab5-399e-4c33-9a59-9689dcb79b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:54.527045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0d0b2ab5-399e-4c33-9a59-9689dcb79b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:54.634880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b2ab5-399e-4c33-9a59-9689dcb79b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:54.636337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b2ab5-399e-4c33-9a59-9689dcb79b3f service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:54.646274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:54.655130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 503/1011] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:15:56.840835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f4adc-073d-4dec-81c0-0045c307433e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:56.841437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f4adc-073d-4dec-81c0-0045c307433e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:56.841437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f4adc-073d-4dec-81c0-0045c307433e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:56.842695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 509/1012] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:15:56.847833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bacafc-6b9f-4e09-b4d4-83e602028a5e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:56.848300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bacafc-6b9f-4e09-b4d4-83e602028a5e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:56.848400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bacafc-6b9f-4e09-b4d4-83e602028a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:56.915394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-78bacafc-6b9f-4e09-b4d4-83e602028a5e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:15:56.916535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-78bacafc-6b9f-4e09-b4d4-83e602028a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:57.009978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 504/1013] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:15:57.298591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f4adc-073d-4dec-81c0-0045c307433e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:15:57.300111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-799f4adc-073d-4dec-81c0-0045c307433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:57.339524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:57.406190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f4adc-073d-4dec-81c0-0045c307433e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:15:57.477527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9141f30-2606-4f73-a85e-99b8ad198d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:15:57.478124 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9141f30-2606-4f73-a85e-99b8ad198d5d service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:15:57.478737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9141f30-2606-4f73-a85e-99b8ad198d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:15:57.479006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9141f30-2606-4f73-a85e-99b8ad198d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:15:57.480603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f9141f30-2606-4f73-a85e-99b8ad198d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:15:57.561336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:15:57.587569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9141f30-2606-4f73-a85e-99b8ad198d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:15:57.589019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9141f30-2606-4f73-a85e-99b8ad198d5d service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:15:57.607242 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 510/1014] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:00.196683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774aa9bc-dd01-4ef0-bbfe-a2d1108c040d None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:00.197166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774aa9bc-dd01-4ef0-bbfe-a2d1108c040d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:00.197270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774aa9bc-dd01-4ef0-bbfe-a2d1108c040d None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:00.198556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 505/1015] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:00.203836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78508a-049b-4da3-8f55-642e83e45fa6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:00.204272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78508a-049b-4da3-8f55-642e83e45fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:00.204404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78508a-049b-4da3-8f55-642e83e45fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:00.269524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8e78508a-049b-4da3-8f55-642e83e45fa6 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:00.270770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8e78508a-049b-4da3-8f55-642e83e45fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:00.365929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 511/1016] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:00.654814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774aa9bc-dd01-4ef0-bbfe-a2d1108c040d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:00.656381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-774aa9bc-dd01-4ef0-bbfe-a2d1108c040d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:00.734888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:00.762642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774aa9bc-dd01-4ef0-bbfe-a2d1108c040d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:00.834806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667945-d39c-4b67-a334-3e6aa75ed45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:00.835474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667945-d39c-4b67-a334-3e6aa75ed45e service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:00.835751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667945-d39c-4b67-a334-3e6aa75ed45e service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:00.836021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667945-d39c-4b67-a334-3e6aa75ed45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:00.837637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f0667945-d39c-4b67-a334-3e6aa75ed45e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:00.943591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667945-d39c-4b67-a334-3e6aa75ed45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:00.945461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667945-d39c-4b67-a334-3e6aa75ed45e service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:00.963450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 506/1017] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:05.076343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6548c-c6aa-4868-a64b-91b003e45022 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:05.076860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6548c-c6aa-4868-a64b-91b003e45022 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:05.076860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6548c-c6aa-4868-a64b-91b003e45022 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:05.078235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 512/1018] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:05.083251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bb429-2178-4f86-abdc-32127488e6f8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:05.083633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bb429-2178-4f86-abdc-32127488e6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:05.083799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bb429-2178-4f86-abdc-32127488e6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:05.089937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:05.151707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-249bb429-2178-4f86-abdc-32127488e6f8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:05.153018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-249bb429-2178-4f86-abdc-32127488e6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:05.247007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 507/1019] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:05.534458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6548c-c6aa-4868-a64b-91b003e45022 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:05.535961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d1f6548c-c6aa-4868-a64b-91b003e45022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:05.596703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:05.651321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6548c-c6aa-4868-a64b-91b003e45022 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:05.726264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cc452-d987-4bb1-a989-430cfb3c4f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:05.726865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cc452-d987-4bb1-a989-430cfb3c4f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:05.727134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cc452-d987-4bb1-a989-430cfb3c4f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:05.727395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cc452-d987-4bb1-a989-430cfb3c4f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:05.729598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5a7cc452-d987-4bb1-a989-430cfb3c4f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:05.821002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:05.837371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cc452-d987-4bb1-a989-430cfb3c4f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:05.838798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cc452-d987-4bb1-a989-430cfb3c4f87 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:05.857014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 513/1020] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:16:08.596688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc89e-9cce-4702-8c80-7705bf52dc0b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:08.597204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc89e-9cce-4702-8c80-7705bf52dc0b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:08.597204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc89e-9cce-4702-8c80-7705bf52dc0b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:08.598595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 508/1021] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:08.603809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0edc-cf8e-43e1-8b2d-449976b2de5b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:08.604219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0edc-cf8e-43e1-8b2d-449976b2de5b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:08.604378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0edc-cf8e-43e1-8b2d-449976b2de5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:08.669641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9b5a0edc-cf8e-43e1-8b2d-449976b2de5b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:08.670819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9b5a0edc-cf8e-43e1-8b2d-449976b2de5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:08.765609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 514/1022] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:09.049341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bc89e-9cce-4702-8c80-7705bf52dc0b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:09.050947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c84bc89e-9cce-4702-8c80-7705bf52dc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:09.138509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:09.156630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bc89e-9cce-4702-8c80-7705bf52dc0b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:09.232557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd27ef-3f88-422d-a653-094658fb7782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:09.233159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd27ef-3f88-422d-a653-094658fb7782 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:09.233434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd27ef-3f88-422d-a653-094658fb7782 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:09.233696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd27ef-3f88-422d-a653-094658fb7782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:09.235686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0ecd27ef-3f88-422d-a653-094658fb7782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:09.351390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd27ef-3f88-422d-a653-094658fb7782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:09.353375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd27ef-3f88-422d-a653-094658fb7782 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:09.371163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 509/1023] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:13.524325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bd721-a273-49bf-97e0-518478100c9d None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:13.524924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bd721-a273-49bf-97e0-518478100c9d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:13.524924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bd721-a273-49bf-97e0-518478100c9d None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:13.526245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 515/1024] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:13.531508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a7849-9249-4a73-a2ea-4a0c43c5f848 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:13.531937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a7849-9249-4a73-a2ea-4a0c43c5f848 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:13.532105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a7849-9249-4a73-a2ea-4a0c43c5f848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:13.548408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:13.599423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-618a7849-9249-4a73-a2ea-4a0c43c5f848 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:13.601360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-618a7849-9249-4a73-a2ea-4a0c43c5f848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:13.700261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 510/1025] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:13.979867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bd721-a273-49bf-97e0-518478100c9d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:13.981426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-331bd721-a273-49bf-97e0-518478100c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:14.038277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:14.083468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bd721-a273-49bf-97e0-518478100c9d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:14.157512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d7c37-7531-4c9f-a07e-9f9fdae3e5d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:14.158106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d7c37-7531-4c9f-a07e-9f9fdae3e5d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:14.158379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d7c37-7531-4c9f-a07e-9f9fdae3e5d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:14.158644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d7c37-7531-4c9f-a07e-9f9fdae3e5d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:14.160206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8c0d7c37-7531-4c9f-a07e-9f9fdae3e5d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:14.274670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:14.278404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d7c37-7531-4c9f-a07e-9f9fdae3e5d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:14.280399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d7c37-7531-4c9f-a07e-9f9fdae3e5d3 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:14.298938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 516/1026] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:19.165970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d946-cebd-4ed5-b21e-eb2f658c10bb None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:19.166880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d946-cebd-4ed5-b21e-eb2f658c10bb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:19.167889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d946-cebd-4ed5-b21e-eb2f658c10bb None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:19.168840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 511/1027] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:19.177179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4a18a-58e5-4f50-934d-dccef0dac385 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:19.177762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4a18a-58e5-4f50-934d-dccef0dac385 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:19.177762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4a18a-58e5-4f50-934d-dccef0dac385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:19.312657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a0e4a18a-58e5-4f50-934d-dccef0dac385 None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:19.313923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a0e4a18a-58e5-4f50-934d-dccef0dac385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:19.432650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 517/1028] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:23.679208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0de6d3-bbee-4c82-ace9-68db960f59fa None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:23.680047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0de6d3-bbee-4c82-ace9-68db960f59fa None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:23.680336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0de6d3-bbee-4c82-ace9-68db960f59fa None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:23.682101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 512/1029] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:23.687151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40412d-908f-4854-bd4b-b704e46d5a5d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:23.687927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40412d-908f-4854-bd4b-b704e46d5a5d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:23.688411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40412d-908f-4854-bd4b-b704e46d5a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:23.698635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=59 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:23.744807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:23.769428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ac40412d-908f-4854-bd4b-b704e46d5a5d None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:23.770625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ac40412d-908f-4854-bd4b-b704e46d5a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:23.892190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 518/1030] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:16:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:25.055836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327430-32e2-4516-92fb-c741b025b45b None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:25.056606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327430-32e2-4516-92fb-c741b025b45b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:25.056606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327430-32e2-4516-92fb-c741b025b45b None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:25.057935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 513/1031] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:25.062662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854159e-f087-4be5-bc95-5d1ae4c5af87 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:25.063089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854159e-f087-4be5-bc95-5d1ae4c5af87 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:25.063182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854159e-f087-4be5-bc95-5d1ae4c5af87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:25.139211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a854159e-f087-4be5-bc95-5d1ae4c5af87 None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:25.140431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a854159e-f087-4be5-bc95-5d1ae4c5af87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:25.181159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:25.247513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 519/1032] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:16:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:16:28.831844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92f5e0-99f1-4669-bbea-641e73c76bb0 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:28.832773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92f5e0-99f1-4669-bbea-641e73c76bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:28.833202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92f5e0-99f1-4669-bbea-641e73c76bb0 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:28.834683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 514/1033] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:28.840000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc9590-bfe7-420a-9448-f58c818e7d9f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:28.840788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc9590-bfe7-420a-9448-f58c818e7d9f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:28.841228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc9590-bfe7-420a-9448-f58c818e7d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:28.916770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-13dc9590-bfe7-420a-9448-f58c818e7d9f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:28.918100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13dc9590-bfe7-420a-9448-f58c818e7d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:29.024321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 520/1034] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:29.324500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73229-bcfc-4ad2-aeb3-c68065225c26 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:29.325280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73229-bcfc-4ad2-aeb3-c68065225c26 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:29.326305 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a73229-bcfc-4ad2-aeb3-c68065225c26 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:29.328645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 515/1035] 10.222.0.48 () {54 vars in 961 bytes} [Thu Oct 27 13:16:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:29.337240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abd37-ce57-4809-a174-96625337ef09 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:29.337840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abd37-ce57-4809-a174-96625337ef09 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:29.338479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abd37-ce57-4809-a174-96625337ef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:29.365446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:29.429676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7b3abd37-ce57-4809-a174-96625337ef09 None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:29.431061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7b3abd37-ce57-4809-a174-96625337ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:29.568823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 521/1036] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Oct 27 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:16:29.576406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73229-bcfc-4ad2-aeb3-c68065225c26 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:29.577971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-70a73229-bcfc-4ad2-aeb3-c68065225c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:29.754050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a73229-bcfc-4ad2-aeb3-c68065225c26 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:29.834206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:29.879436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698f0e9-eb46-4cfa-9212-9b2c461b1c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:29.880321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698f0e9-eb46-4cfa-9212-9b2c461b1c99 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:29.880567 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698f0e9-eb46-4cfa-9212-9b2c461b1c99 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:29.880970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698f0e9-eb46-4cfa-9212-9b2c461b1c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:29.882872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1698f0e9-eb46-4cfa-9212-9b2c461b1c99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:29.986795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698f0e9-eb46-4cfa-9212-9b2c461b1c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:29.988415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698f0e9-eb46-4cfa-9212-9b2c461b1c99 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:30.010900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 516/1037] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:32.423723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f375d59-8895-4746-83ce-2ee7b5411a4f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:32.424335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f375d59-8895-4746-83ce-2ee7b5411a4f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:32.424335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f375d59-8895-4746-83ce-2ee7b5411a4f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:32.432632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc23db-c242-42ac-85b2-6833c942315a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:32.433228 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc23db-c242-42ac-85b2-6833c942315a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:32.433228 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc23db-c242-42ac-85b2-6833c942315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:32.437655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 522/1038] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:32.505168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-20dc23db-c242-42ac-85b2-6833c942315a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:32.507125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-20dc23db-c242-42ac-85b2-6833c942315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:32.561580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:32.648015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 517/1039] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:32.939387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f375d59-8895-4746-83ce-2ee7b5411a4f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:32.940860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7f375d59-8895-4746-83ce-2ee7b5411a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:33.017442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:33.117610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f375d59-8895-4746-83ce-2ee7b5411a4f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:33.239243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf558f-f10d-46e3-9056-a49a3ce0964d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:33.239857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf558f-f10d-46e3-9056-a49a3ce0964d service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:33.240135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf558f-f10d-46e3-9056-a49a3ce0964d service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:33.240437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf558f-f10d-46e3-9056-a49a3ce0964d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:33.242034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fbaf558f-f10d-46e3-9056-a49a3ce0964d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:33.266354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:33.335262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf558f-f10d-46e3-9056-a49a3ce0964d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:33.336888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf558f-f10d-46e3-9056-a49a3ce0964d service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:33.355663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 523/1040] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:35.850683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa173064-c79f-4bb9-87da-bba32e05de3e None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:35.851213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa173064-c79f-4bb9-87da-bba32e05de3e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:35.851213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa173064-c79f-4bb9-87da-bba32e05de3e None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:35.852779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 518/1041] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:35.857836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1a668-7384-4fdb-8b18-843c5f0b0599 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:35.858270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1a668-7384-4fdb-8b18-843c5f0b0599 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:35.858354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1a668-7384-4fdb-8b18-843c5f0b0599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:35.928972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-06b1a668-7384-4fdb-8b18-843c5f0b0599 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:35.930179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-06b1a668-7384-4fdb-8b18-843c5f0b0599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:36.029245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:36.040714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 524/1042] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:16:36.320025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4aec-4fb9-4ee2-a86f-9fe7b98fe803 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:36.320326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4aec-4fb9-4ee2-a86f-9fe7b98fe803 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:36.320599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4aec-4fb9-4ee2-a86f-9fe7b98fe803 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:36.322078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 519/1043] 10.222.0.48 () {54 vars in 961 bytes} [Thu Oct 27 13:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:36.327348 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e26b-b310-4d82-a71e-e8f28c0fbae9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:36.327699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e26b-b310-4d82-a71e-e8f28c0fbae9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:36.328068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e26b-b310-4d82-a71e-e8f28c0fbae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:36.405786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-00f9e26b-b310-4d82-a71e-e8f28c0fbae9 None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:36.406970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-00f9e26b-b310-4d82-a71e-e8f28c0fbae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:36.526921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 525/1044] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Oct 27 13:16:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:36.535953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4aec-4fb9-4ee2-a86f-9fe7b98fe803 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:36.537585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-043a4aec-4fb9-4ee2-a86f-9fe7b98fe803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:36.694591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4aec-4fb9-4ee2-a86f-9fe7b98fe803 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:36.720407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:36.817025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304da6d-d836-4d27-8d65-f2c49bc311c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:36.817550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304da6d-d836-4d27-8d65-f2c49bc311c0 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:36.817821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304da6d-d836-4d27-8d65-f2c49bc311c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:36.818085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304da6d-d836-4d27-8d65-f2c49bc311c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:36.819651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0304da6d-d836-4d27-8d65-f2c49bc311c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:36.912874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304da6d-d836-4d27-8d65-f2c49bc311c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:36.914459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304da6d-d836-4d27-8d65-f2c49bc311c0 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:36.932980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 520/1045] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:16:37.072031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1964-73b9-4c26-b6ea-32cad3bc1d63 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:37.072644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1964-73b9-4c26-b6ea-32cad3bc1d63 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:37.072644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1964-73b9-4c26-b6ea-32cad3bc1d63 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:37.074042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 526/1046] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:16:37.079707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d956ee-59b3-47e1-be9c-15c02efdac60 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:37.080005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d956ee-59b3-47e1-be9c-15c02efdac60 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:37.080359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d956ee-59b3-47e1-be9c-15c02efdac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:37.117807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:37.168331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e9d956ee-59b3-47e1-be9c-15c02efdac60 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:37.169921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e9d956ee-59b3-47e1-be9c-15c02efdac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:37.290602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 521/1047] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:37.626537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234a322-b6d8-4b5a-991e-75b2b2e77414 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:37.626537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234a322-b6d8-4b5a-991e-75b2b2e77414 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:37.626537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234a322-b6d8-4b5a-991e-75b2b2e77414 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:37.628644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 527/1048] 10.222.0.48 () {54 vars in 963 bytes} [Thu Oct 27 13:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:37.633827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48ce0-1b52-4e86-b619-b3b26cc3694e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:37.634114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48ce0-1b52-4e86-b619-b3b26cc3694e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:37.634391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48ce0-1b52-4e86-b619-b3b26cc3694e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:37.718434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4ae48ce0-1b52-4e86-b619-b3b26cc3694e None None] MFA Rules not processed for user `80a57dd6e99c4cac9e6bcd1413afd5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:37.719750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4ae48ce0-1b52-4e86-b619-b3b26cc3694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:37.771642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:37.804837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 522/1049] 10.222.0.48 () {58 vars in 1060 bytes} [Thu Oct 27 13:16:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:16:37.812905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234a322-b6d8-4b5a-991e-75b2b2e77414 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:37.814440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9234a322-b6d8-4b5a-991e-75b2b2e77414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:37.869946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:37.934846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234a322-b6d8-4b5a-991e-75b2b2e77414 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:38.038976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abd6ac-4096-44cd-b183-32a64b18ead4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:38.039829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd6ac-4096-44cd-b183-32a64b18ead4 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:38.040194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd6ac-4096-44cd-b183-32a64b18ead4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:38.040581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd6ac-4096-44cd-b183-32a64b18ead4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:38.042668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-63abd6ac-4096-44cd-b183-32a64b18ead4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:38.197577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:38.214004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abd6ac-4096-44cd-b183-32a64b18ead4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:38.215882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abd6ac-4096-44cd-b183-32a64b18ead4 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:38.248138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 528/1050] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:39.187241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791fbd7-79d0-4d9c-9938-06416e6d55b4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:39.187742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791fbd7-79d0-4d9c-9938-06416e6d55b4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:39.187742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791fbd7-79d0-4d9c-9938-06416e6d55b4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:39.189068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 523/1051] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:39.195076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e1de-3aa4-4091-a1ac-60b95147a615 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:39.195358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e1de-3aa4-4091-a1ac-60b95147a615 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:39.195628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e1de-3aa4-4091-a1ac-60b95147a615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:39.261018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-25e0e1de-3aa4-4091-a1ac-60b95147a615 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:39.262207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-25e0e1de-3aa4-4091-a1ac-60b95147a615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:39.358305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 529/1052] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:39.648989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791fbd7-79d0-4d9c-9938-06416e6d55b4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:39.650580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3791fbd7-79d0-4d9c-9938-06416e6d55b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:39.811780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791fbd7-79d0-4d9c-9938-06416e6d55b4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:39.866311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:39.928748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c297b-c19a-4f64-8cbd-16adb34f1205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:39.929354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c297b-c19a-4f64-8cbd-16adb34f1205 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:39.929624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c297b-c19a-4f64-8cbd-16adb34f1205 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:39.929886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c297b-c19a-4f64-8cbd-16adb34f1205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:39.931452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b22c297b-c19a-4f64-8cbd-16adb34f1205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:40.024270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c297b-c19a-4f64-8cbd-16adb34f1205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:40.025892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c297b-c19a-4f64-8cbd-16adb34f1205 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:40.045315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 524/1053] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:42.260305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff3ff2-ab44-444c-b22a-ccb92f6fd911 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:42.260834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff3ff2-ab44-444c-b22a-ccb92f6fd911 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:42.261060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff3ff2-ab44-444c-b22a-ccb92f6fd911 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:42.262621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 530/1054] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:42.268380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc4304-1a0b-4269-a5b4-3414f967f17a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:42.268874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc4304-1a0b-4269-a5b4-3414f967f17a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:42.269101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc4304-1a0b-4269-a5b4-3414f967f17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:42.338141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f3fc4304-1a0b-4269-a5b4-3414f967f17a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:42.339844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f3fc4304-1a0b-4269-a5b4-3414f967f17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:42.347061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:42.452025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 525/1055] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:42.736579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff3ff2-ab44-444c-b22a-ccb92f6fd911 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:42.738604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d1ff3ff2-ab44-444c-b22a-ccb92f6fd911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:42.802402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:42.890584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff3ff2-ab44-444c-b22a-ccb92f6fd911 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:43.005745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69610a-b0e7-439c-9995-c7e0563d12a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:43.006599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69610a-b0e7-439c-9995-c7e0563d12a1 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:43.006870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69610a-b0e7-439c-9995-c7e0563d12a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:43.007132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69610a-b0e7-439c-9995-c7e0563d12a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:43.008705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-eb69610a-b0e7-439c-9995-c7e0563d12a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:43.027600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:43.100374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69610a-b0e7-439c-9995-c7e0563d12a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:43.101948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69610a-b0e7-439c-9995-c7e0563d12a1 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:43.120567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 531/1056] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:45.317165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7782106-8254-44d9-83bb-2622c064a530 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:45.317753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7782106-8254-44d9-83bb-2622c064a530 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:45.317753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7782106-8254-44d9-83bb-2622c064a530 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:45.319224 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 526/1057] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:45.324386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e83d-a598-4e16-826f-8059eae0a2ee None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:45.324815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e83d-a598-4e16-826f-8059eae0a2ee None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:45.324904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e83d-a598-4e16-826f-8059eae0a2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:45.391796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3027e83d-a598-4e16-826f-8059eae0a2ee None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:45.393011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3027e83d-a598-4e16-826f-8059eae0a2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:45.488429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:45.500721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 532/1058] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:45.783146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7782106-8254-44d9-83bb-2622c064a530 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:45.784830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b7782106-8254-44d9-83bb-2622c064a530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:45.921833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:45.933097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7782106-8254-44d9-83bb-2622c064a530 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:46.046905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72393fcc-6394-4c0f-9b26-a9b3ef2cb670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:46.047511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393fcc-6394-4c0f-9b26-a9b3ef2cb670 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:46.047789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393fcc-6394-4c0f-9b26-a9b3ef2cb670 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:46.048087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393fcc-6394-4c0f-9b26-a9b3ef2cb670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:46.049674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-72393fcc-6394-4c0f-9b26-a9b3ef2cb670 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:46.146406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72393fcc-6394-4c0f-9b26-a9b3ef2cb670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:46.147997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72393fcc-6394-4c0f-9b26-a9b3ef2cb670 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:46.158117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:46.167294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 527/1059] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:48.394830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ecf3b-f32f-4c52-8e09-f56cef73199b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:48.395347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ecf3b-f32f-4c52-8e09-f56cef73199b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:48.395347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ecf3b-f32f-4c52-8e09-f56cef73199b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:48.396800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 533/1060] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:48.403245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69a3ef-50e0-4187-a828-d1f45a15debc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:48.403651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69a3ef-50e0-4187-a828-d1f45a15debc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:48.403779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69a3ef-50e0-4187-a828-d1f45a15debc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:48.474454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-be69a3ef-50e0-4187-a828-d1f45a15debc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:48.475590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-be69a3ef-50e0-4187-a828-d1f45a15debc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:48.575943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 528/1061] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:48.865994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ecf3b-f32f-4c52-8e09-f56cef73199b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:48.867541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5f3ecf3b-f32f-4c52-8e09-f56cef73199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:49.026186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ecf3b-f32f-4c52-8e09-f56cef73199b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:49.103700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:49.152040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01622c-5956-4872-a3d6-9c7fc4911125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:49.152671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01622c-5956-4872-a3d6-9c7fc4911125 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:49.152948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01622c-5956-4872-a3d6-9c7fc4911125 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:49.153223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01622c-5956-4872-a3d6-9c7fc4911125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:49.154807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4f01622c-5956-4872-a3d6-9c7fc4911125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:49.243265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01622c-5956-4872-a3d6-9c7fc4911125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:49.244887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01622c-5956-4872-a3d6-9c7fc4911125 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:49.263245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 534/1062] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:51.493148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7d71e-01a1-4435-a82f-ddb2ed29813a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:51.493793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7d71e-01a1-4435-a82f-ddb2ed29813a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:51.493793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7d71e-01a1-4435-a82f-ddb2ed29813a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:51.495017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 529/1063] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:51.500215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916649bd-bfb1-426c-aab9-734b5bd5988a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:51.500690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916649bd-bfb1-426c-aab9-734b5bd5988a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:51.500792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916649bd-bfb1-426c-aab9-734b5bd5988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:51.567603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-916649bd-bfb1-426c-aab9-734b5bd5988a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:51.568818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-916649bd-bfb1-426c-aab9-734b5bd5988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:51.606652 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:51.673352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 535/1064] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:51.959713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7d71e-01a1-4435-a82f-ddb2ed29813a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:51.961261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-74d7d71e-01a1-4435-a82f-ddb2ed29813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:52.038457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:52.107424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7d71e-01a1-4435-a82f-ddb2ed29813a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:52.222615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cb9ac-dc9f-4ee8-8726-2a92db569fe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:52.223220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb9ac-dc9f-4ee8-8726-2a92db569fe5 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:52.223499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb9ac-dc9f-4ee8-8726-2a92db569fe5 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:52.223771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb9ac-dc9f-4ee8-8726-2a92db569fe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:52.225424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-999cb9ac-dc9f-4ee8-8726-2a92db569fe5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:52.265403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:52.317941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cb9ac-dc9f-4ee8-8726-2a92db569fe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:52.319528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cb9ac-dc9f-4ee8-8726-2a92db569fe5 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:52.337535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 530/1065] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:53.423894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73629398-e2cd-4cb5-9522-d7830e70b895 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:53.424545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73629398-e2cd-4cb5-9522-d7830e70b895 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:53.424545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73629398-e2cd-4cb5-9522-d7830e70b895 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:53.425844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 536/1066] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:53.430976 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2566b4e-c612-4f86-a162-90e60745580b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:53.431406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2566b4e-c612-4f86-a162-90e60745580b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:53.431515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2566b4e-c612-4f86-a162-90e60745580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:53.496366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d2566b4e-c612-4f86-a162-90e60745580b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:53.497553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d2566b4e-c612-4f86-a162-90e60745580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:53.598592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 531/1067] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:53.903144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa408ab6-98b0-40fd-8357-2a7fb16b75f1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:53.903425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa408ab6-98b0-40fd-8357-2a7fb16b75f1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:53.903707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa408ab6-98b0-40fd-8357-2a7fb16b75f1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:53.905087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 537/1068] 10.222.0.48 () {54 vars in 962 bytes} [Thu Oct 27 13:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:53.910433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b14306-0e26-4a81-96e5-1e557636955c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:53.910825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b14306-0e26-4a81-96e5-1e557636955c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:53.911158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b14306-0e26-4a81-96e5-1e557636955c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:53.926699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:53.987649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a5b14306-0e26-4a81-96e5-1e557636955c None None] MFA Rules not processed for user `2d4ce3e02e5945ffbbde389398b146c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:53.988853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a5b14306-0e26-4a81-96e5-1e557636955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:54.062997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 532/1069] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Oct 27 13:16:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:16:54.071309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa408ab6-98b0-40fd-8357-2a7fb16b75f1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:54.072943 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fa408ab6-98b0-40fd-8357-2a7fb16b75f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:54.176413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa408ab6-98b0-40fd-8357-2a7fb16b75f1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:54.235071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:54.250935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eab64-718e-4ac1-b37e-05dcea6c3c6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d4ce3e02e5945ffbbde389398b146c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:54.251538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eab64-718e-4ac1-b37e-05dcea6c3c6d service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:54.251811 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eab64-718e-4ac1-b37e-05dcea6c3c6d service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:54.252070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eab64-718e-4ac1-b37e-05dcea6c3c6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:54.254261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d73eab64-718e-4ac1-b37e-05dcea6c3c6d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:54.365093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73eab64-718e-4ac1-b37e-05dcea6c3c6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:54.366524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73eab64-718e-4ac1-b37e-05dcea6c3c6d service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:54.384965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 538/1070] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:54.535303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c884d9-dc64-43ed-abdf-48498ca05bd5 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:54.535830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c884d9-dc64-43ed-abdf-48498ca05bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:54.535830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c884d9-dc64-43ed-abdf-48498ca05bd5 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:54.537220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 533/1071] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:54.542655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63be4d1-2ce2-4326-a4e6-b62d8e6f85d3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:54.543088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63be4d1-2ce2-4326-a4e6-b62d8e6f85d3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:54.543314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63be4d1-2ce2-4326-a4e6-b62d8e6f85d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:54.621738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c63be4d1-2ce2-4326-a4e6-b62d8e6f85d3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:54.622950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c63be4d1-2ce2-4326-a4e6-b62d8e6f85d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:54.667166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:54.726480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 539/1072] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:55.018509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c884d9-dc64-43ed-abdf-48498ca05bd5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:55.020043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-70c884d9-dc64-43ed-abdf-48498ca05bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:55.125726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:55.145008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73483b-2518-4950-b811-f8ad587a9b15 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:55.145290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73483b-2518-4950-b811-f8ad587a9b15 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:55.145556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73483b-2518-4950-b811-f8ad587a9b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:55.171698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c884d9-dc64-43ed-abdf-48498ca05bd5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:55.213403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6c73483b-2518-4950-b811-f8ad587a9b15 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:55.214599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6c73483b-2518-4950-b811-f8ad587a9b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:55.286209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:55.286814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:55.287088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:55.287376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:55.289011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:55.304515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 540/1073] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:16:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:16:55.352944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa3bf5-6b86-41fb-9449-7076eb68b19f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:55.353225 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa3bf5-6b86-41fb-9449-7076eb68b19f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:55.353497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa3bf5-6b86-41fb-9449-7076eb68b19f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:55.354960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 541/1074] 10.222.0.48 () {54 vars in 943 bytes} [Thu Oct 27 13:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:55.360924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572fb8-c166-466f-b368-6735caeb35e2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:55.361200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572fb8-c166-466f-b368-6735caeb35e2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:55.362269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572fb8-c166-466f-b368-6735caeb35e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:55.362390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:55.383051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:55.384681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:55.403789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 534/1075] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:16:55.405252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:55.444060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-25572fb8-c166-466f-b368-6735caeb35e2 None None] MFA Rules not processed for user `edb1a2c4a51f46c1be39baf0927eff4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:55.445280 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-25572fb8-c166-466f-b368-6735caeb35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:55.562982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 542/1076] 10.222.0.48 () {58 vars in 1040 bytes} [Thu Oct 27 13:16:55 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:55.573610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:55.574744 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:55.669594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87badc20-92a6-4a95-a69c-c9410e396835 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:55.773576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:55.774968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478f7b4-67e4-4f8c-892c-35abd4d3598b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:55.775603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f7b4-67e4-4f8c-892c-35abd4d3598b service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:55.775870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f7b4-67e4-4f8c-892c-35abd4d3598b service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:55.776469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f7b4-67e4-4f8c-892c-35abd4d3598b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:55.778508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9478f7b4-67e4-4f8c-892c-35abd4d3598b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:55.862632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478f7b4-67e4-4f8c-892c-35abd4d3598b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:55.864216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478f7b4-67e4-4f8c-892c-35abd4d3598b service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:55.872120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 535/1077] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:16:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:16:57.631046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882349ff-723c-40e2-8003-eed4959adb05 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:57.631475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882349ff-723c-40e2-8003-eed4959adb05 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:57.631588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882349ff-723c-40e2-8003-eed4959adb05 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:57.633089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 543/1078] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:16:57.638223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bc40a-3e20-4758-9660-c676494452e4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:57.638676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bc40a-3e20-4758-9660-c676494452e4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:57.638784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bc40a-3e20-4758-9660-c676494452e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:57.716463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f00bc40a-3e20-4758-9660-c676494452e4 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:16:57.717679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f00bc40a-3e20-4758-9660-c676494452e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:57.814620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:57.822309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 536/1079] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:16:58.110014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882349ff-723c-40e2-8003-eed4959adb05 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:16:58.111539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-882349ff-723c-40e2-8003-eed4959adb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:58.215791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:58.281412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882349ff-723c-40e2-8003-eed4959adb05 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:16:58.406621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41c7ca-13d4-4dbe-8ea3-82e1961bccca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:16:58.407224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41c7ca-13d4-4dbe-8ea3-82e1961bccca service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:16:58.407494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41c7ca-13d4-4dbe-8ea3-82e1961bccca service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:16:58.407757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41c7ca-13d4-4dbe-8ea3-82e1961bccca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:16:58.409877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ec41c7ca-13d4-4dbe-8ea3-82e1961bccca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:16:58.468499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:16:58.511939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41c7ca-13d4-4dbe-8ea3-82e1961bccca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:16:58.513660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41c7ca-13d4-4dbe-8ea3-82e1961bccca service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:16:58.532860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 544/1080] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:17:00.729720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfd54-6eb7-48b9-9b70-163882f2a62a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:00.730300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfd54-6eb7-48b9-9b70-163882f2a62a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:00.730442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfd54-6eb7-48b9-9b70-163882f2a62a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:00.732350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 537/1081] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:00.738994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971c011-9948-4578-901b-c833c04a6938 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:00.739657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971c011-9948-4578-901b-c833c04a6938 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:00.739779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971c011-9948-4578-901b-c833c04a6938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:00.809934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7971c011-9948-4578-901b-c833c04a6938 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:00.811182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7971c011-9948-4578-901b-c833c04a6938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:00.920368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 545/1082] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:01.214379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cfd54-6eb7-48b9-9b70-163882f2a62a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:01.214379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b07cfd54-6eb7-48b9-9b70-163882f2a62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:01.376355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cfd54-6eb7-48b9-9b70-163882f2a62a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:01.459977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:01.512106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e14536-3ddc-49fc-8676-dd222a0d1038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:01.512737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14536-3ddc-49fc-8676-dd222a0d1038 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:01.513012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14536-3ddc-49fc-8676-dd222a0d1038 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:01.513279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14536-3ddc-49fc-8676-dd222a0d1038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:01.514868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90e14536-3ddc-49fc-8676-dd222a0d1038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:01.618391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e14536-3ddc-49fc-8676-dd222a0d1038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:01.619972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e14536-3ddc-49fc-8676-dd222a0d1038 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:01.642135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 538/1083] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:03.842155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8854-25c9-476e-9b96-77129c91ece9 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:03.842705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8854-25c9-476e-9b96-77129c91ece9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:03.842705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c8854-25c9-476e-9b96-77129c91ece9 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:03.843996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 546/1084] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:03.849450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaf97b-8db2-42df-a630-af662a55c54b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:03.849900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaf97b-8db2-42df-a630-af662a55c54b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:03.850148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdaf97b-8db2-42df-a630-af662a55c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:03.925617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9bdaf97b-8db2-42df-a630-af662a55c54b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:03.926748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9bdaf97b-8db2-42df-a630-af662a55c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:03.963216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:04.041788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 539/1085] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:04.331334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8854-25c9-476e-9b96-77129c91ece9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:04.332854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bb3c8854-25c9-476e-9b96-77129c91ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:04.370802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:04.489523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c8854-25c9-476e-9b96-77129c91ece9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:04.618694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:04.626141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca5274-256c-4a7f-acc9-d852b6700111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:04.627362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca5274-256c-4a7f-acc9-d852b6700111 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:04.627634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca5274-256c-4a7f-acc9-d852b6700111 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:04.628041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca5274-256c-4a7f-acc9-d852b6700111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:04.629651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-18ca5274-256c-4a7f-acc9-d852b6700111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:04.727672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca5274-256c-4a7f-acc9-d852b6700111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:04.729508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca5274-256c-4a7f-acc9-d852b6700111 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:04.749460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 547/1086] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:14.073246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=30 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:14.746948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=48 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:18.698279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df739c-e6d6-401b-a633-127c7ed382b5 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:18.698855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df739c-e6d6-401b-a633-127c7ed382b5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:18.698855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df739c-e6d6-401b-a633-127c7ed382b5 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:18.700152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 540/1087] 10.222.0.48 () {54 vars in 918 bytes} [Thu Oct 27 13:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:18.705886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009f187-f21a-411f-a696-6784b26f9c53 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:18.706290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009f187-f21a-411f-a696-6784b26f9c53 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:18.706424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009f187-f21a-411f-a696-6784b26f9c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:18.780977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0009f187-f21a-411f-a696-6784b26f9c53 None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:18.782204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0009f187-f21a-411f-a696-6784b26f9c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:18.885959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 548/1088] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Oct 27 13:17:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:18.899927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e0d4-3820-4c8d-b0d4-56af41be6b25 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:18.900211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e0d4-3820-4c8d-b0d4-56af41be6b25 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:18.900513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2e0d4-3820-4c8d-b0d4-56af41be6b25 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:18.901845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 541/1089] 10.222.0.48 () {54 vars in 918 bytes} [Thu Oct 27 13:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:18.906293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9efe8-433e-419a-a93f-106b0a2aed1c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:18.906582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9efe8-433e-419a-a93f-106b0a2aed1c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:18.906860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9efe8-433e-419a-a93f-106b0a2aed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:18.990400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:18.992988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b0b9efe8-433e-419a-a93f-106b0a2aed1c None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:18.994724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b0b9efe8-433e-419a-a93f-106b0a2aed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:19.106129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 549/1090] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Oct 27 13:17:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:17:21.712470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91f1ca-ddad-4b0f-a1ae-e110c8f0a976 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:21.713039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91f1ca-ddad-4b0f-a1ae-e110c8f0a976 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:21.713039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91f1ca-ddad-4b0f-a1ae-e110c8f0a976 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:21.714371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 542/1091] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:21.719859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6003711-e92a-4d98-a5e6-9d8a867d7219 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:21.720307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6003711-e92a-4d98-a5e6-9d8a867d7219 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:21.720495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6003711-e92a-4d98-a5e6-9d8a867d7219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:21.786551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b6003711-e92a-4d98-a5e6-9d8a867d7219 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:21.787740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b6003711-e92a-4d98-a5e6-9d8a867d7219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:21.842607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:21.885398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 550/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:17:22.180577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91f1ca-ddad-4b0f-a1ae-e110c8f0a976 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:22.182195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac91f1ca-ddad-4b0f-a1ae-e110c8f0a976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:22.326048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91f1ca-ddad-4b0f-a1ae-e110c8f0a976 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:22.397975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:22.437738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b145ef4-9751-4c06-9c9c-0ce1a221e4cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:22.438262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145ef4-9751-4c06-9c9c-0ce1a221e4cc service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:22.438533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145ef4-9751-4c06-9c9c-0ce1a221e4cc service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:22.438797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145ef4-9751-4c06-9c9c-0ce1a221e4cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:22.440394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4b145ef4-9751-4c06-9c9c-0ce1a221e4cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:22.527478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b145ef4-9751-4c06-9c9c-0ce1a221e4cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:22.529058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b145ef4-9751-4c06-9c9c-0ce1a221e4cc service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:22.547160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 543/1093] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:23.666697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ed23e-38c9-4df5-86a6-3789c8d9ca07 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:23.667175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ed23e-38c9-4df5-86a6-3789c8d9ca07 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:23.667268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ed23e-38c9-4df5-86a6-3789c8d9ca07 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:23.668593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 551/1094] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:23.673753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abba3-5c52-4850-ba3c-e3578d5999ee None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:23.674171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abba3-5c52-4850-ba3c-e3578d5999ee None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:23.674294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abba3-5c52-4850-ba3c-e3578d5999ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:23.747975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ef0abba3-5c52-4850-ba3c-e3578d5999ee None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:23.749189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ef0abba3-5c52-4850-ba3c-e3578d5999ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:23.776370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:23.844330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 544/1095] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:17:24.006300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ed23e-38c9-4df5-86a6-3789c8d9ca07 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:24.007721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-552ed23e-38c9-4df5-86a6-3789c8d9ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:24.149020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ed23e-38c9-4df5-86a6-3789c8d9ca07 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:24.189036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:24.261021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5320d00-0db9-4b1c-a249-0dc7ea445e12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:24.261554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5320d00-0db9-4b1c-a249-0dc7ea445e12 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:24.261826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5320d00-0db9-4b1c-a249-0dc7ea445e12 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:24.262091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5320d00-0db9-4b1c-a249-0dc7ea445e12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:24.263641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e5320d00-0db9-4b1c-a249-0dc7ea445e12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:24.350656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5320d00-0db9-4b1c-a249-0dc7ea445e12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:24.352222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5320d00-0db9-4b1c-a249-0dc7ea445e12 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:24.370174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 552/1096] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:25.477985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b849-4721-4e71-8ebc-f520a5444e57 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:25.478604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b849-4721-4e71-8ebc-f520a5444e57 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:25.478604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b849-4721-4e71-8ebc-f520a5444e57 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:25.480031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 545/1097] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:25.485393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab590-b91c-44f4-8850-11548e372c62 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:25.485786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab590-b91c-44f4-8850-11548e372c62 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:25.485935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab590-b91c-44f4-8850-11548e372c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:25.538628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:25.560629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0c9ab590-b91c-44f4-8850-11548e372c62 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:25.561814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0c9ab590-b91c-44f4-8850-11548e372c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:25.658200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 553/1098] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:25.807778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b849-4721-4e71-8ebc-f520a5444e57 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:25.809229 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9ab3b849-4721-4e71-8ebc-f520a5444e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:25.951728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b849-4721-4e71-8ebc-f520a5444e57 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:25.963486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:26.062435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb49d9-89d6-42cc-a596-458379c9aca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:26.062954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb49d9-89d6-42cc-a596-458379c9aca4 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:26.063225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb49d9-89d6-42cc-a596-458379c9aca4 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:26.063488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb49d9-89d6-42cc-a596-458379c9aca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:26.065082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-becb49d9-89d6-42cc-a596-458379c9aca4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:26.152243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb49d9-89d6-42cc-a596-458379c9aca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:26.153852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb49d9-89d6-42cc-a596-458379c9aca4 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:26.172320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 546/1099] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:27.260325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ccf0-21ba-4692-b591-5bf0252cc71c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:27.260849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ccf0-21ba-4692-b591-5bf0252cc71c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:27.260849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ccf0-21ba-4692-b591-5bf0252cc71c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:27.262169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 554/1100] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:27.267475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a8233-f8c5-4e3b-9609-067bd554d754 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:27.267886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a8233-f8c5-4e3b-9609-067bd554d754 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:27.267998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a8233-f8c5-4e3b-9609-067bd554d754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:27.288217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:27.342753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3c3a8233-f8c5-4e3b-9609-067bd554d754 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:27.344513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3c3a8233-f8c5-4e3b-9609-067bd554d754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:27.442799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 547/1101] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:27.591561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785ccf0-21ba-4692-b591-5bf0252cc71c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:27.593159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5785ccf0-21ba-4692-b591-5bf0252cc71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:27.701613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:27.738115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785ccf0-21ba-4692-b591-5bf0252cc71c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:27.849065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01531517-e274-4fa8-a190-37aeac73d8c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:27.849675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01531517-e274-4fa8-a190-37aeac73d8c3 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:27.849953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01531517-e274-4fa8-a190-37aeac73d8c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:27.850226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01531517-e274-4fa8-a190-37aeac73d8c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:27.851812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-01531517-e274-4fa8-a190-37aeac73d8c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:27.928048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:27.942267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01531517-e274-4fa8-a190-37aeac73d8c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:27.943853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01531517-e274-4fa8-a190-37aeac73d8c3 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:27.961770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 555/1102] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:29.047617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd88d9-6e23-477e-8d8e-24d445dadad3 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:29.048150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd88d9-6e23-477e-8d8e-24d445dadad3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:29.048150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd88d9-6e23-477e-8d8e-24d445dadad3 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:29.049589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 548/1103] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:29.054334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145c76d-4b90-435d-9542-1fad4a42835e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:29.054721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145c76d-4b90-435d-9542-1fad4a42835e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:29.054861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145c76d-4b90-435d-9542-1fad4a42835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:29.127808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6145c76d-4b90-435d-9542-1fad4a42835e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:29.129277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6145c76d-4b90-435d-9542-1fad4a42835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:29.222021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 556/1104] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:29.374313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd88d9-6e23-477e-8d8e-24d445dadad3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:29.375803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-64bd88d9-6e23-477e-8d8e-24d445dadad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:29.460392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:29.523479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd88d9-6e23-477e-8d8e-24d445dadad3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:29.634237 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d0658-fac4-41e1-abd5-de657d249749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:29.634836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d0658-fac4-41e1-abd5-de657d249749 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:29.635106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d0658-fac4-41e1-abd5-de657d249749 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:29.635370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d0658-fac4-41e1-abd5-de657d249749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:29.636978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8c6d0658-fac4-41e1-abd5-de657d249749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:29.685193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:29.726518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d0658-fac4-41e1-abd5-de657d249749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:29.728075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d0658-fac4-41e1-abd5-de657d249749 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:29.745761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 549/1105] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:30.788220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cf1c-6eb4-4cf5-98ba-66402bf1d85e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:30.788715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cf1c-6eb4-4cf5-98ba-66402bf1d85e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:30.788715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cf1c-6eb4-4cf5-98ba-66402bf1d85e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:30.790034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 557/1106] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:30.795000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1f5a4-17b6-471e-a7cc-faf30e4e0dbd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:30.795384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1f5a4-17b6-471e-a7cc-faf30e4e0dbd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:30.795548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1f5a4-17b6-471e-a7cc-faf30e4e0dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:30.870762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-52c1f5a4-17b6-471e-a7cc-faf30e4e0dbd None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:30.871951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-52c1f5a4-17b6-471e-a7cc-faf30e4e0dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:30.965691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 550/1107] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:31.120591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7cf1c-6eb4-4cf5-98ba-66402bf1d85e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:31.122089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-22f7cf1c-6eb4-4cf5-98ba-66402bf1d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:31.170373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:31.265037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7cf1c-6eb4-4cf5-98ba-66402bf1d85e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:31.374079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8e2c9-fe7d-4dd7-be8e-d96ec009c6d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:31.374607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8e2c9-fe7d-4dd7-be8e-d96ec009c6d1 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:31.374879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8e2c9-fe7d-4dd7-be8e-d96ec009c6d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:31.375146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8e2c9-fe7d-4dd7-be8e-d96ec009c6d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:31.376746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5ca8e2c9-fe7d-4dd7-be8e-d96ec009c6d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:31.383732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:31.464990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8e2c9-fe7d-4dd7-be8e-d96ec009c6d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:31.466574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8e2c9-fe7d-4dd7-be8e-d96ec009c6d1 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:31.485062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 558/1108] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:32.527546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b104f7-ea2f-4532-b6a7-c6234185a66a None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:32.528154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b104f7-ea2f-4532-b6a7-c6234185a66a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:32.528154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b104f7-ea2f-4532-b6a7-c6234185a66a None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:32.529457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 551/1109] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:32.536349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19308449-f6c6-4dc3-bf24-d748e0184171 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:32.536775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19308449-f6c6-4dc3-bf24-d748e0184171 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:32.536945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19308449-f6c6-4dc3-bf24-d748e0184171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:32.612384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-19308449-f6c6-4dc3-bf24-d748e0184171 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:32.613709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-19308449-f6c6-4dc3-bf24-d748e0184171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:32.682607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:32.711085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 559/1110] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:32.862108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b104f7-ea2f-4532-b6a7-c6234185a66a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:32.863615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e0b104f7-ea2f-4532-b6a7-c6234185a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:32.881084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:33.005280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b104f7-ea2f-4532-b6a7-c6234185a66a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:33.092092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:33.117477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f3dc9-2b79-425d-a2c7-973d1b4f56a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:33.118012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f3dc9-2b79-425d-a2c7-973d1b4f56a8 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:33.118281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f3dc9-2b79-425d-a2c7-973d1b4f56a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:33.118542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f3dc9-2b79-425d-a2c7-973d1b4f56a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:33.120108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7a7f3dc9-2b79-425d-a2c7-973d1b4f56a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:33.206529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f3dc9-2b79-425d-a2c7-973d1b4f56a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:33.208112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f3dc9-2b79-425d-a2c7-973d1b4f56a8 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:33.225923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 552/1111] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:34.259457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1d89-37a8-46a2-bff3-6e2f3a0a9aca None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:34.259902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1d89-37a8-46a2-bff3-6e2f3a0a9aca None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:34.260013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1d89-37a8-46a2-bff3-6e2f3a0a9aca None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:34.261347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 560/1112] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:34.266160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7210-7321-4deb-afb9-aba9b0407a89 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:34.266618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7210-7321-4deb-afb9-aba9b0407a89 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:34.266716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7210-7321-4deb-afb9-aba9b0407a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:34.340949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-660f7210-7321-4deb-afb9-aba9b0407a89 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:34.342132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-660f7210-7321-4deb-afb9-aba9b0407a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:34.383343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:34.436846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 553/1113] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:34.586675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af1d89-37a8-46a2-bff3-6e2f3a0a9aca None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:34.588089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d4af1d89-37a8-46a2-bff3-6e2f3a0a9aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:34.731556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af1d89-37a8-46a2-bff3-6e2f3a0a9aca None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:34.785084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:34.842200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619556dc-9991-4297-8a95-d3a116b57db1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:34.843343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619556dc-9991-4297-8a95-d3a116b57db1 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:34.843631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619556dc-9991-4297-8a95-d3a116b57db1 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:34.843879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619556dc-9991-4297-8a95-d3a116b57db1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:34.845471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-619556dc-9991-4297-8a95-d3a116b57db1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:34.929963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619556dc-9991-4297-8a95-d3a116b57db1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:34.931526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619556dc-9991-4297-8a95-d3a116b57db1 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:34.949227 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 561/1114] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:35.983950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b73bd7-a576-4d21-b07d-191d3d13f3f7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:35.984493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b73bd7-a576-4d21-b07d-191d3d13f3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:35.984493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b73bd7-a576-4d21-b07d-191d3d13f3f7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:35.985781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 554/1115] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:35.990558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5d649-3daf-486a-9b64-5b6a7fcb8ff8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:35.990957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5d649-3daf-486a-9b64-5b6a7fcb8ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:35.991099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5d649-3daf-486a-9b64-5b6a7fcb8ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:36.065867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-46d5d649-3daf-486a-9b64-5b6a7fcb8ff8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:36.067057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-46d5d649-3daf-486a-9b64-5b6a7fcb8ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:36.069061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:36.162062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 562/1116] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:36.311680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b73bd7-a576-4d21-b07d-191d3d13f3f7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:36.313112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-35b73bd7-a576-4d21-b07d-191d3d13f3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:36.456654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b73bd7-a576-4d21-b07d-191d3d13f3f7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:36.483679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:36.567795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369bde7-3008-45a8-95cd-7c9a438e774a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:36.568408 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369bde7-3008-45a8-95cd-7c9a438e774a service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:36.568682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369bde7-3008-45a8-95cd-7c9a438e774a service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:36.568944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369bde7-3008-45a8-95cd-7c9a438e774a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:36.570506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f369bde7-3008-45a8-95cd-7c9a438e774a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:36.659218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369bde7-3008-45a8-95cd-7c9a438e774a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:36.660824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369bde7-3008-45a8-95cd-7c9a438e774a service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:36.678301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 555/1117] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:38.735565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536557b-493f-4514-83a9-a03c0dea9ae5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:38.736157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536557b-493f-4514-83a9-a03c0dea9ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:38.736157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536557b-493f-4514-83a9-a03c0dea9ae5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:38.737466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 563/1118] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:38.742690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2e7f-8f9a-43ec-9cd0-db939c231f24 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:38.743124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2e7f-8f9a-43ec-9cd0-db939c231f24 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:38.743231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2e7f-8f9a-43ec-9cd0-db939c231f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:38.785696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:38.811573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-837d2e7f-8f9a-43ec-9cd0-db939c231f24 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:38.812807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-837d2e7f-8f9a-43ec-9cd0-db939c231f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:38.908664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 556/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:39.201879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536557b-493f-4514-83a9-a03c0dea9ae5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:39.203380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7536557b-493f-4514-83a9-a03c0dea9ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:39.305061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536557b-493f-4514-83a9-a03c0dea9ae5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:39.336611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:39.379182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7089d6-af9e-4911-97d5-7762cfef5d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:39.379786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7089d6-af9e-4911-97d5-7762cfef5d58 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:39.380065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7089d6-af9e-4911-97d5-7762cfef5d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:39.380364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7089d6-af9e-4911-97d5-7762cfef5d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:39.381944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6e7089d6-af9e-4911-97d5-7762cfef5d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:39.490070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7089d6-af9e-4911-97d5-7762cfef5d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:39.491432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7089d6-af9e-4911-97d5-7762cfef5d58 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:39.509767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 564/1120] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:41.822290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648d1a6-e813-4cb1-bcf2-4a375ee48593 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:41.822853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648d1a6-e813-4cb1-bcf2-4a375ee48593 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:41.822853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648d1a6-e813-4cb1-bcf2-4a375ee48593 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:41.824159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 557/1121] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:41.829473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f93fd-98cd-44d9-ab97-1f5a85c0b0ea None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:41.829898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f93fd-98cd-44d9-ab97-1f5a85c0b0ea None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:41.830084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f93fd-98cd-44d9-ab97-1f5a85c0b0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:41.895248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ac9f93fd-98cd-44d9-ab97-1f5a85c0b0ea None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:41.896418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ac9f93fd-98cd-44d9-ab97-1f5a85c0b0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:41.909294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:41.991134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 565/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:42.274878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648d1a6-e813-4cb1-bcf2-4a375ee48593 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:42.276469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1648d1a6-e813-4cb1-bcf2-4a375ee48593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:42.379521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648d1a6-e813-4cb1-bcf2-4a375ee48593 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:42.395494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:42.455404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff57b4-9049-40da-982e-375bf541c5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:42.456001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff57b4-9049-40da-982e-375bf541c5ab service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:42.456297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff57b4-9049-40da-982e-375bf541c5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:42.456564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff57b4-9049-40da-982e-375bf541c5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:42.458191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-71ff57b4-9049-40da-982e-375bf541c5ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:42.565570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff57b4-9049-40da-982e-375bf541c5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:42.566998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff57b4-9049-40da-982e-375bf541c5ab service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:42.584649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 558/1123] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:45.055936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2c560-2e24-4e0a-827b-ee5c854f50b5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:45.056524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2c560-2e24-4e0a-827b-ee5c854f50b5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:45.056524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2c560-2e24-4e0a-827b-ee5c854f50b5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:45.057778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 566/1124] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:45.063154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39a12-eb37-4940-b058-64bf0ec8342c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:45.063568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39a12-eb37-4940-b058-64bf0ec8342c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:45.063702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39a12-eb37-4940-b058-64bf0ec8342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:45.112297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:45.130412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-51d39a12-eb37-4940-b058-64bf0ec8342c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:45.131602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-51d39a12-eb37-4940-b058-64bf0ec8342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:45.226315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 559/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:45.518266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2c560-2e24-4e0a-827b-ee5c854f50b5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:45.519777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e1c2c560-2e24-4e0a-827b-ee5c854f50b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:45.620539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2c560-2e24-4e0a-827b-ee5c854f50b5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:45.660298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:45.694304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae1309-f5a7-402e-bc5c-c5a1fff6298b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:45.694907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1309-f5a7-402e-bc5c-c5a1fff6298b service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:45.695182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1309-f5a7-402e-bc5c-c5a1fff6298b service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:45.695452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1309-f5a7-402e-bc5c-c5a1fff6298b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:45.697069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-94ae1309-f5a7-402e-bc5c-c5a1fff6298b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:45.804782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae1309-f5a7-402e-bc5c-c5a1fff6298b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:45.806131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae1309-f5a7-402e-bc5c-c5a1fff6298b service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:45.824056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 567/1126] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:48.064889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d7ec6-0f8a-4ef0-a0c2-381b9233becf None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:48.065413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d7ec6-0f8a-4ef0-a0c2-381b9233becf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:48.065413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d7ec6-0f8a-4ef0-a0c2-381b9233becf None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:48.066740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 560/1127] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:48.071936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118384d-ec81-49be-abf4-4b032d722ae2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:48.072343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118384d-ec81-49be-abf4-4b032d722ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:48.072522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118384d-ec81-49be-abf4-4b032d722ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:48.138443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4118384d-ec81-49be-abf4-4b032d722ae2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:48.139642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4118384d-ec81-49be-abf4-4b032d722ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:48.160109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:48.235329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 568/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:48.522535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d7ec6-0f8a-4ef0-a0c2-381b9233becf None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:48.524096 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-903d7ec6-0f8a-4ef0-a0c2-381b9233becf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:48.626287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d7ec6-0f8a-4ef0-a0c2-381b9233becf None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:48.646395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:48.698623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2afda-8c93-4e44-9086-3535d457707e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:48.699218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2afda-8c93-4e44-9086-3535d457707e service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:48.699490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2afda-8c93-4e44-9086-3535d457707e service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:48.699751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2afda-8c93-4e44-9086-3535d457707e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:48.701386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5fb2afda-8c93-4e44-9086-3535d457707e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:48.808443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2afda-8c93-4e44-9086-3535d457707e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:48.809791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2afda-8c93-4e44-9086-3535d457707e service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:48.827789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 561/1129] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:51.037376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73457fda-8d7b-482d-b287-671e123ff2dc None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:51.037939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73457fda-8d7b-482d-b287-671e123ff2dc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:51.037939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73457fda-8d7b-482d-b287-671e123ff2dc None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:51.039164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 569/1130] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:51.044442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5693f5e-c497-4efc-a953-b702fe97dbdd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:51.044840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5693f5e-c497-4efc-a953-b702fe97dbdd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:51.044988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5693f5e-c497-4efc-a953-b702fe97dbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:51.107141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:51.109831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d5693f5e-c497-4efc-a953-b702fe97dbdd None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:51.111585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d5693f5e-c497-4efc-a953-b702fe97dbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:51.208569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 562/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:51.495707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73457fda-8d7b-482d-b287-671e123ff2dc None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:51.497769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-73457fda-8d7b-482d-b287-671e123ff2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:51.597898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73457fda-8d7b-482d-b287-671e123ff2dc None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:51.646384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:51.670425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546188c2-f8e0-4ac6-95fc-83dd583f97be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:51.671044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546188c2-f8e0-4ac6-95fc-83dd583f97be service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:51.671299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546188c2-f8e0-4ac6-95fc-83dd583f97be service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:51.671565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546188c2-f8e0-4ac6-95fc-83dd583f97be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:51.673156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-546188c2-f8e0-4ac6-95fc-83dd583f97be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:51.779324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546188c2-f8e0-4ac6-95fc-83dd583f97be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:51.780704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546188c2-f8e0-4ac6-95fc-83dd583f97be service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:51.799003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 570/1132] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:53.998801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57019727-1c74-4567-95c6-d79d60918708 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:53.999318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57019727-1c74-4567-95c6-d79d60918708 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:53.999318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57019727-1c74-4567-95c6-d79d60918708 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:54.000648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 563/1133] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:54.006146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a6dca-3724-44fe-b9a8-c58fc129d09b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:54.006560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a6dca-3724-44fe-b9a8-c58fc129d09b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:54.006724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a6dca-3724-44fe-b9a8-c58fc129d09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:54.070967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6a6a6dca-3724-44fe-b9a8-c58fc129d09b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:54.072149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6a6a6dca-3724-44fe-b9a8-c58fc129d09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:54.101074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:54.167599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 571/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:54.451739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57019727-1c74-4567-95c6-d79d60918708 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:54.453301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-57019727-1c74-4567-95c6-d79d60918708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:54.555242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57019727-1c74-4567-95c6-d79d60918708 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:54.582968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:54.630093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926873e-5e09-47f2-bd33-aabf6498bd3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:54.630705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926873e-5e09-47f2-bd33-aabf6498bd3b service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:54.630977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926873e-5e09-47f2-bd33-aabf6498bd3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:54.631242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926873e-5e09-47f2-bd33-aabf6498bd3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:54.632847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3926873e-5e09-47f2-bd33-aabf6498bd3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:54.740490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926873e-5e09-47f2-bd33-aabf6498bd3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:54.741814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926873e-5e09-47f2-bd33-aabf6498bd3b service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:54.759919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 564/1135] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:17:56.966268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddecc-7a47-4061-95e2-736b3a58c89e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:56.966775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddecc-7a47-4061-95e2-736b3a58c89e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:56.966775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bddecc-7a47-4061-95e2-736b3a58c89e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:56.968053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 572/1136] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:56.973437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a7f57-f5af-4e09-92d8-2e052d4b5bdf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:56.973835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a7f57-f5af-4e09-92d8-2e052d4b5bdf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:56.974013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a7f57-f5af-4e09-92d8-2e052d4b5bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:57.039200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dd6a7f57-f5af-4e09-92d8-2e052d4b5bdf None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:57.040428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dd6a7f57-f5af-4e09-92d8-2e052d4b5bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:57.049134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:57.138922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 565/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:17:57.423512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddecc-7a47-4061-95e2-736b3a58c89e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:17:57.425068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c5bddecc-7a47-4061-95e2-736b3a58c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:57.525983 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bddecc-7a47-4061-95e2-736b3a58c89e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:17:57.580448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:17:57.599237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c7d79-b43e-4eaa-833f-246d08995d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:17:57.599844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c7d79-b43e-4eaa-833f-246d08995d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:57.600120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c7d79-b43e-4eaa-833f-246d08995d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:57.600418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c7d79-b43e-4eaa-833f-246d08995d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:57.601996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b73c7d79-b43e-4eaa-833f-246d08995d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:17:57.710542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c7d79-b43e-4eaa-833f-246d08995d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:17:57.711864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c7d79-b43e-4eaa-833f-246d08995d30 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:17:57.730665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 573/1138] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:17:59.919131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece9a1-8178-4cb7-9376-d6d50cf7afdc None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:59.919682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece9a1-8178-4cb7-9376-d6d50cf7afdc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:59.919682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece9a1-8178-4cb7-9376-d6d50cf7afdc None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:59.921026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 566/1139] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:17:59.926243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1533d09-165d-4de8-b01e-17ccc8069b72 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:17:59.926678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1533d09-165d-4de8-b01e-17ccc8069b72 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:17:59.926773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1533d09-165d-4de8-b01e-17ccc8069b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:17:59.991326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e1533d09-165d-4de8-b01e-17ccc8069b72 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:17:59.992555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e1533d09-165d-4de8-b01e-17ccc8069b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:00.030902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:00.087319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 574/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:00.367604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece9a1-8178-4cb7-9376-d6d50cf7afdc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:00.369151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-10ece9a1-8178-4cb7-9376-d6d50cf7afdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:00.470995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece9a1-8178-4cb7-9376-d6d50cf7afdc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:00.508209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:00.543888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f2e7d-34c5-4cfe-9513-2956eb5ce35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:00.544503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f2e7d-34c5-4cfe-9513-2956eb5ce35b service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:00.544779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f2e7d-34c5-4cfe-9513-2956eb5ce35b service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:00.545048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f2e7d-34c5-4cfe-9513-2956eb5ce35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:00.546620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac8f2e7d-34c5-4cfe-9513-2956eb5ce35b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:00.657233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f2e7d-34c5-4cfe-9513-2956eb5ce35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:00.658645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f2e7d-34c5-4cfe-9513-2956eb5ce35b service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:00.676651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 567/1141] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:02.861883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bbfe3-faab-4bd3-93cc-bd0b19038282 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:02.862420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bbfe3-faab-4bd3-93cc-bd0b19038282 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:02.862534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bbfe3-faab-4bd3-93cc-bd0b19038282 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:02.863829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 575/1142] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:02.869066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c48a2-8909-448c-ad7c-ee1f0886bd5b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:02.869472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c48a2-8909-448c-ad7c-ee1f0886bd5b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:02.869600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c48a2-8909-448c-ad7c-ee1f0886bd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:02.934316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-036c48a2-8909-448c-ad7c-ee1f0886bd5b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:02.935481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-036c48a2-8909-448c-ad7c-ee1f0886bd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:02.952590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:03.029994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 568/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:03.314685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bbfe3-faab-4bd3-93cc-bd0b19038282 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:03.316270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-201bbfe3-faab-4bd3-93cc-bd0b19038282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:03.418131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bbfe3-faab-4bd3-93cc-bd0b19038282 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:03.482926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:03.489490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ee9a0-040c-417f-8795-233c67dc16ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:03.490086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ee9a0-040c-417f-8795-233c67dc16ee service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:03.490359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ee9a0-040c-417f-8795-233c67dc16ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:03.490626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ee9a0-040c-417f-8795-233c67dc16ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:03.492634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-661ee9a0-040c-417f-8795-233c67dc16ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:03.600210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ee9a0-040c-417f-8795-233c67dc16ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:03.601710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ee9a0-040c-417f-8795-233c67dc16ee service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:03.620286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 576/1144] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:05.761917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d7e7c-e34e-4ac9-900b-9b7bfd046f81 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:05.762476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d7e7c-e34e-4ac9-900b-9b7bfd046f81 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:05.762476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d7e7c-e34e-4ac9-900b-9b7bfd046f81 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:05.763858 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 569/1145] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:05.769053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7a99-b313-4068-b29a-55261001b664 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:05.769490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7a99-b313-4068-b29a-55261001b664 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:05.769604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7a99-b313-4068-b29a-55261001b664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:05.835039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d22d7a99-b313-4068-b29a-55261001b664 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:05.836267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d22d7a99-b313-4068-b29a-55261001b664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:05.887478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:05.938108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 577/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:06.217277 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d7e7c-e34e-4ac9-900b-9b7bfd046f81 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:06.218787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f78d7e7c-e34e-4ac9-900b-9b7bfd046f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:06.321448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d7e7c-e34e-4ac9-900b-9b7bfd046f81 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:06.368781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:06.395082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec815ed-a0a3-47d4-ae8f-76c061b4217b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:06.395677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec815ed-a0a3-47d4-ae8f-76c061b4217b service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:06.395948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec815ed-a0a3-47d4-ae8f-76c061b4217b service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:06.396214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec815ed-a0a3-47d4-ae8f-76c061b4217b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:06.397795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9ec815ed-a0a3-47d4-ae8f-76c061b4217b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:06.502387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec815ed-a0a3-47d4-ae8f-76c061b4217b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:06.503733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec815ed-a0a3-47d4-ae8f-76c061b4217b service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:06.522038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 570/1147] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:15.936254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=22 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:16.519810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=66 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:26.335726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eee7be-ec87-48f9-bf29-a9eca77b0c9e None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:26.336340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eee7be-ec87-48f9-bf29-a9eca77b0c9e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:26.336340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eee7be-ec87-48f9-bf29-a9eca77b0c9e None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:26.337614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 578/1148] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:26.343547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77086b-d68a-4b26-88ea-eafe6a9381aa None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:26.343947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77086b-d68a-4b26-88ea-eafe6a9381aa None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:26.344092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77086b-d68a-4b26-88ea-eafe6a9381aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:26.412899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3e77086b-d68a-4b26-88ea-eafe6a9381aa None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:26.414088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e77086b-d68a-4b26-88ea-eafe6a9381aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:26.508757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 571/1149] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:26.811342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c961ffd-ae16-427c-b4af-a88006982492 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:26.811623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c961ffd-ae16-427c-b4af-a88006982492 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:26.811896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c961ffd-ae16-427c-b4af-a88006982492 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:26.813233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 579/1150] 10.222.0.48 () {54 vars in 962 bytes} [Thu Oct 27 13:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:26.818348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc29fd-939f-49ef-b1a2-27149a803337 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:26.818631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc29fd-939f-49ef-b1a2-27149a803337 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:26.818905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc29fd-939f-49ef-b1a2-27149a803337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:26.893810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-91fc29fd-939f-49ef-b1a2-27149a803337 None None] MFA Rules not processed for user `2d4ce3e02e5945ffbbde389398b146c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:26.895001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-91fc29fd-939f-49ef-b1a2-27149a803337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:26.915698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:26.971182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 572/1151] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Oct 27 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:26.978852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c961ffd-ae16-427c-b4af-a88006982492 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:26.980375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2c961ffd-ae16-427c-b4af-a88006982492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:27.081826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c961ffd-ae16-427c-b4af-a88006982492 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:27.156896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6aa9f-c803-40a7-ad82-b828320d81b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d4ce3e02e5945ffbbde389398b146c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:27.157504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6aa9f-c803-40a7-ad82-b828320d81b5 service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:27.157782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6aa9f-c803-40a7-ad82-b828320d81b5 service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:27.158050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6aa9f-c803-40a7-ad82-b828320d81b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:27.159623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-85c6aa9f-c803-40a7-ad82-b828320d81b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:27.213885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:27.274641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6aa9f-c803-40a7-ad82-b828320d81b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:27.276072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6aa9f-c803-40a7-ad82-b828320d81b5 service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:27.294882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 580/1152] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:18:29.529527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea251e0-a4d0-4ec8-aedf-51ffe295c7f7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:29.530049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea251e0-a4d0-4ec8-aedf-51ffe295c7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:29.530049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea251e0-a4d0-4ec8-aedf-51ffe295c7f7 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:29.531399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 573/1153] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:29.536818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd6e3-45fc-4e72-b9f1-ba152e603148 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:29.537284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd6e3-45fc-4e72-b9f1-ba152e603148 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:29.537386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd6e3-45fc-4e72-b9f1-ba152e603148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:29.602521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-404cd6e3-45fc-4e72-b9f1-ba152e603148 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:29.603693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-404cd6e3-45fc-4e72-b9f1-ba152e603148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:29.698360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 581/1154] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:29.976932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea251e0-a4d0-4ec8-aedf-51ffe295c7f7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:29.978453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2ea251e0-a4d0-4ec8-aedf-51ffe295c7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:30.099807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea251e0-a4d0-4ec8-aedf-51ffe295c7f7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:30.149301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:30.192389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb85c71-b35d-4657-8536-576091cbc935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:30.192984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85c71-b35d-4657-8536-576091cbc935 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:30.193257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85c71-b35d-4657-8536-576091cbc935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:30.193522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85c71-b35d-4657-8536-576091cbc935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:30.194150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85c71-b35d-4657-8536-576091cbc935 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:30.195230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-aeb85c71-b35d-4657-8536-576091cbc935 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:18:30.195230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:18:30.195230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:18:30.196380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85c71-b35d-4657-8536-576091cbc935 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:30.214771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 574/1155] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Oct 27 13:18:29 2022] GET /identity/v3/projects => generated 2553 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:18:32.362823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c803e1-ae4c-471f-8b91-68739e9835b2 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:32.363369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c803e1-ae4c-471f-8b91-68739e9835b2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:32.363469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c803e1-ae4c-471f-8b91-68739e9835b2 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:32.364849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 582/1156] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:32.370135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afec7f0-c982-4d98-b94c-05b8a04c6c2e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:32.370615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afec7f0-c982-4d98-b94c-05b8a04c6c2e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:32.370720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afec7f0-c982-4d98-b94c-05b8a04c6c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:32.439602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4afec7f0-c982-4d98-b94c-05b8a04c6c2e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:32.440837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4afec7f0-c982-4d98-b94c-05b8a04c6c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:32.537037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 575/1157] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:32.830570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c803e1-ae4c-471f-8b91-68739e9835b2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:32.832083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-41c803e1-ae4c-471f-8b91-68739e9835b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:32.842159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:32.976709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c803e1-ae4c-471f-8b91-68739e9835b2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:33.055733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:33.088190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5175ea-12e6-4dd9-8df7-666343ecc793 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:33.088743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5175ea-12e6-4dd9-8df7-666343ecc793 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:33.089016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5175ea-12e6-4dd9-8df7-666343ecc793 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:33.089280 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5175ea-12e6-4dd9-8df7-666343ecc793 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:33.090899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4a5175ea-12e6-4dd9-8df7-666343ecc793 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:33.178171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5175ea-12e6-4dd9-8df7-666343ecc793 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:33.179727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5175ea-12e6-4dd9-8df7-666343ecc793 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:33.197322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 583/1158] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:35.500424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326531e0-c7c3-4382-8d2c-a8a60ede2f23 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:35.500994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326531e0-c7c3-4382-8d2c-a8a60ede2f23 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:35.500994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326531e0-c7c3-4382-8d2c-a8a60ede2f23 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:35.502281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 576/1159] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:35.507744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4776-f31c-4eca-8215-deaab699b0e3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:35.508186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4776-f31c-4eca-8215-deaab699b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:35.508307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4776-f31c-4eca-8215-deaab699b0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:35.572589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-573f4776-f31c-4eca-8215-deaab699b0e3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:35.573777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-573f4776-f31c-4eca-8215-deaab699b0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:35.620322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:35.669633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 584/1160] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:35.954681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326531e0-c7c3-4382-8d2c-a8a60ede2f23 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:35.956201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-326531e0-c7c3-4382-8d2c-a8a60ede2f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:35.980368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:36.058379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326531e0-c7c3-4382-8d2c-a8a60ede2f23 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:36.129364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd56c2-ac46-4f94-9b5a-d3eb75db8772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:36.129957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd56c2-ac46-4f94-9b5a-d3eb75db8772 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:36.130227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd56c2-ac46-4f94-9b5a-d3eb75db8772 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:36.130489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd56c2-ac46-4f94-9b5a-d3eb75db8772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:36.132044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-00dd56c2-ac46-4f94-9b5a-d3eb75db8772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:36.205256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:36.239538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd56c2-ac46-4f94-9b5a-d3eb75db8772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:36.240974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd56c2-ac46-4f94-9b5a-d3eb75db8772 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:36.259109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 577/1161] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:38.442194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6235-2ff1-4a4f-889f-f86371a351d9 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:38.442752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6235-2ff1-4a4f-889f-f86371a351d9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:38.442752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6235-2ff1-4a4f-889f-f86371a351d9 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:38.444048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 585/1162] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:38.449250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0d92e-61c2-4e3c-ad78-f2a6f83acab6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:38.449633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0d92e-61c2-4e3c-ad78-f2a6f83acab6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:38.450316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0d92e-61c2-4e3c-ad78-f2a6f83acab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:38.515455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a3a0d92e-61c2-4e3c-ad78-f2a6f83acab6 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:38.516607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a3a0d92e-61c2-4e3c-ad78-f2a6f83acab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:38.611876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 578/1163] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:38.897194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e6235-2ff1-4a4f-889f-f86371a351d9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:38.898717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a88e6235-2ff1-4a4f-889f-f86371a351d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:38.999683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e6235-2ff1-4a4f-889f-f86371a351d9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:39.070128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:39.071708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bea47a-0752-43d6-8d29-1810f3513cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:39.072375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bea47a-0752-43d6-8d29-1810f3513cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:39.072749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bea47a-0752-43d6-8d29-1810f3513cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:39.073201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bea47a-0752-43d6-8d29-1810f3513cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:39.075229 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-58bea47a-0752-43d6-8d29-1810f3513cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:39.183390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bea47a-0752-43d6-8d29-1810f3513cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:39.185322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bea47a-0752-43d6-8d29-1810f3513cf5 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:39.202430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 586/1164] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:41.503076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695918e-2d97-41e3-9f5c-ac0da9eab7b4 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:41.503682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695918e-2d97-41e3-9f5c-ac0da9eab7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:41.503682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695918e-2d97-41e3-9f5c-ac0da9eab7b4 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:41.505229 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 579/1165] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:41.510459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508532a-d9c5-402d-8866-bc016cda10e2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:41.510914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508532a-d9c5-402d-8866-bc016cda10e2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:41.511019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508532a-d9c5-402d-8866-bc016cda10e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:41.575866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f508532a-d9c5-402d-8866-bc016cda10e2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:41.577072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f508532a-d9c5-402d-8866-bc016cda10e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:41.629763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:41.673409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 587/1166] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:41.967315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695918e-2d97-41e3-9f5c-ac0da9eab7b4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:41.968874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6695918e-2d97-41e3-9f5c-ac0da9eab7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:42.004428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:42.070373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695918e-2d97-41e3-9f5c-ac0da9eab7b4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:42.142383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8c40c-de0b-46a5-844e-aeb63eab6271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:42.142983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8c40c-de0b-46a5-844e-aeb63eab6271 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:42.143257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8c40c-de0b-46a5-844e-aeb63eab6271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:42.143522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8c40c-de0b-46a5-844e-aeb63eab6271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:42.145113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5af8c40c-de0b-46a5-844e-aeb63eab6271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:42.223724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:42.251590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8c40c-de0b-46a5-844e-aeb63eab6271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:42.253031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8c40c-de0b-46a5-844e-aeb63eab6271 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:42.272128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 580/1167] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:44.612842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0d519-8bd7-4ae7-8f4b-046b8e3c2e18 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:44.613365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0d519-8bd7-4ae7-8f4b-046b8e3c2e18 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:44.613464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0d519-8bd7-4ae7-8f4b-046b8e3c2e18 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:44.614773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 588/1168] 10.222.0.48 () {54 vars in 927 bytes} [Thu Oct 27 13:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:18:44.620164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd6109-99ad-4ba8-ab80-7a96009408e1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:44.620607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd6109-99ad-4ba8-ab80-7a96009408e1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:44.620734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd6109-99ad-4ba8-ab80-7a96009408e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:44.686559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-67cd6109-99ad-4ba8-ab80-7a96009408e1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:18:44.687730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-67cd6109-99ad-4ba8-ab80-7a96009408e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:44.781543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 581/1169] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Oct 27 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:18:45.068197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0d519-8bd7-4ae7-8f4b-046b8e3c2e18 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:18:45.069833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-27a0d519-8bd7-4ae7-8f4b-046b8e3c2e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:45.171586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0d519-8bd7-4ae7-8f4b-046b8e3c2e18 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:18:45.242708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fb1f7-b7f5-4407-8951-709335548b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:18:45.243310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb1f7-b7f5-4407-8951-709335548b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:18:45.243582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb1f7-b7f5-4407-8951-709335548b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:18:45.243844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fb1f7-b7f5-4407-8951-709335548b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:18:45.245430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-797fb1f7-b7f5-4407-8951-709335548b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:18:45.254670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:45.359149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fb1f7-b7f5-4407-8951-709335548b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:18:45.360527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fb1f7-b7f5-4407-8951-709335548b08 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:18:45.378463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 589/1170] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:18:54.779960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=82 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:18:55.375928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=50 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:20.260255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:20.260889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:20.260889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:20.336037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:20:20.337253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:20.430347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 582/1171] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:20:20.439138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d15546-860c-4033-a8a5-492db2f38b97 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:20.439622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d15546-860c-4033-a8a5-492db2f38b97 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:20.439818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d15546-860c-4033-a8a5-492db2f38b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:20.515555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-34d15546-860c-4033-a8a5-492db2f38b97 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:20:20.516712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-34d15546-860c-4033-a8a5-492db2f38b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:20.610642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 590/1172] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:20:20.615317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:20.616554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:20.628516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084822e0-c68b-4891-8fc4-c7f07f5473ca None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:20.693600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:20.727500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:20.728097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:20.728400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:20.728663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:20.729283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:20:20.729573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:20.730557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:20:20.730557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:20:20.730557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:20:20.731418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:20.742130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 583/1173] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:20:20.752129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d15546-860c-4033-a8a5-492db2f38b97 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:20.761296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d15546-860c-4033-a8a5-492db2f38b97 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:20.821462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:20.854431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:20.855028 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:20.855297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:20.855560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:20.856170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:20.857208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:20:20.857208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:20:20.857208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:20:20.858398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:20.912540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 591/1174] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:20:20 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:20:20.916870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:20.918348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:20.950903 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21d9c9-07ef-4921-8a6c-8017055ed589 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:21.042360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:21.042950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:21.043220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:21.043480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:21.044102 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:21.045638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:20:21.045638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:20:21.045638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:20:21.046814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:21.087428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 584/1175] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:20:20 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:20:21.091555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:21.093092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:21.153795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5290d2-9b9d-422e-934f-936bf7b7270b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:21.237697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:21.249256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:21.249855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:21.250126 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:21.250392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:21.251009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:21.252628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:21.266865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 592/1176] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:20:21.270888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:21.272460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:21.283624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3640d4a-0b93-4135-87a6-145e3e2468d8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:21.309263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:21.378765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a79e2-ddf7-43fb-a12e-d1671de3364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:21.379394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a79e2-ddf7-43fb-a12e-d1671de3364b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:21.379671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a79e2-ddf7-43fb-a12e-d1671de3364b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:21.379939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a79e2-ddf7-43fb-a12e-d1671de3364b admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:21.380596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a79e2-ddf7-43fb-a12e-d1671de3364b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:21.382170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a79e2-ddf7-43fb-a12e-d1671de3364b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:21.394486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 585/1177] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:20:21.398695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:21.400209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:21.433610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6979e6e-bbd8-48b0-a9c0-303aa1b83f56 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:21.523442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec5421-41b7-4736-ab9b-b5de10a23764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:21.524066 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5421-41b7-4736-ab9b-b5de10a23764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:21.524372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5421-41b7-4736-ab9b-b5de10a23764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:21.524638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5421-41b7-4736-ab9b-b5de10a23764 admin admin] PATH_INFO: `/v3/projects/fddd1645636b456d8308a7a4b26d0e6e/users/b110a86087984b2b926bb095cbc7de6f/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:21.547537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec5421-41b7-4736-ab9b-b5de10a23764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddd1645636b456d8308a7a4b26d0e6e, user_id=b110a86087984b2b926bb095cbc7de6f, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:21.550184 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec5421-41b7-4736-ab9b-b5de10a23764 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:21.574483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 593/1178] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:20:21 2022] PUT /identity/v3/projects/fddd1645636b456d8308a7a4b26d0e6e/users/b110a86087984b2b926bb095cbc7de6f/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:20:21.585426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901e8ef-198f-48b2-a965-8b514e2f20b4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:21.585708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901e8ef-198f-48b2-a965-8b514e2f20b4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:21.585979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901e8ef-198f-48b2-a965-8b514e2f20b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:21.643311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2901e8ef-198f-48b2-a965-8b514e2f20b4 None None] MFA Rules not processed for user `b110a86087984b2b926bb095cbc7de6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:20:21.644463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2901e8ef-198f-48b2-a965-8b514e2f20b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:21.720623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 586/1179] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:20:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:20:21.734190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db3654-7220-4040-a403-15cb6446ab32 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:21.734470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db3654-7220-4040-a403-15cb6446ab32 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:21.734736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db3654-7220-4040-a403-15cb6446ab32 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:21.736045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 594/1180] 10.222.0.48 () {54 vars in 963 bytes} [Thu Oct 27 13:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:20:21.740840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849b333-471c-48df-9115-e3229f8ec5c9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:21.741120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849b333-471c-48df-9115-e3229f8ec5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:21.741388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849b333-471c-48df-9115-e3229f8ec5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:21.772307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:21.816807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0849b333-471c-48df-9115-e3229f8ec5c9 None None] MFA Rules not processed for user `80a57dd6e99c4cac9e6bcd1413afd5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:20:21.817989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0849b333-471c-48df-9115-e3229f8ec5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:21.891701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 587/1181] 10.222.0.48 () {58 vars in 1060 bytes} [Thu Oct 27 13:20:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:20:21.899361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db3654-7220-4040-a403-15cb6446ab32 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:21.900939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b7db3654-7220-4040-a403-15cb6446ab32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:21.949960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:22.003933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db3654-7220-4040-a403-15cb6446ab32 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:22.075413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:22.076030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:22.076333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:22.076601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:22.078157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:22.174763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:22.176089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:22.179493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:22.194065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 595/1182] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:20:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:20:22.239162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d1cd-b56f-48a5-b163-a1be7a1afd24 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:22.239441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d1cd-b56f-48a5-b163-a1be7a1afd24 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:22.239705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d1cd-b56f-48a5-b163-a1be7a1afd24 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:22.240981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 588/1183] 10.222.0.48 () {52 vars in 830 bytes} [Thu Oct 27 13:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:20:22.244588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:22.245800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:22.343883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304b6a-fd77-44d1-9ca5-49defe9d70ba service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:22.434515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:22.435053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:22.435351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:22.435628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] PATH_INFO: `/v3/users/b110a86087984b2b926bb095cbc7de6f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:22.448188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b110a86087984b2b926bb095cbc7de6f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:22.449359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:20:22.449359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:20:22.449359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:20:22.450564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:22.460187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:22.514731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b110a86087984b2b926bb095cbc7de6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:22.537144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b110a86087984b2b926bb095cbc7de6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:22.537695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b110a86087984b2b926bb095cbc7de6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:22.551163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b110a86087984b2b926bb095cbc7de6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:22.560002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b110a86087984b2b926bb095cbc7de6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:22.565738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 596/1184] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:20:22 2022] DELETE /identity/v3/users/b110a86087984b2b926bb095cbc7de6f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:20:22.570134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:22.570613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:22.570725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:22.627385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:20:22.628612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:22.703326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:22.715655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 589/1185] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:20:22.725995 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:22.727186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:22.823001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9fd1-db90-4674-a8e9-16791a2d0beb admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:22.878182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:22.899277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf4b1f-6d36-4bf6-bea1-efbc59e6e10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:22.899889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4b1f-6d36-4bf6-bea1-efbc59e6e10a service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:22.900165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4b1f-6d36-4bf6-bea1-efbc59e6e10a service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:22.900465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4b1f-6d36-4bf6-bea1-efbc59e6e10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:22.902058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0cdf4b1f-6d36-4bf6-bea1-efbc59e6e10a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:23.007561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf4b1f-6d36-4bf6-bea1-efbc59e6e10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:23.008957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf4b1f-6d36-4bf6-bea1-efbc59e6e10a service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:23.027166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 597/1186] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:20:23.351561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:20:23.352815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:20:23.364729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7826dd-fb43-4cc8-aa6e-26e67c362d75 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:20:23.458857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:20:23.459481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:20:23.459753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:20:23.460016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] PATH_INFO: `/v3/projects/fddd1645636b456d8308a7a4b26d0e6e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:20:23.469733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fddd1645636b456d8308a7a4b26d0e6e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:20:23.470764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:20:23.470764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:20:23.470764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:20:23.471945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:20:23.528780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fddd1645636b456d8308a7a4b26d0e6e', 'request_id': 'req-fced2939-7200-4677-8caf-2ac999ad4210'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:23.537840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fddd1645636b456d8308a7a4b26d0e6e', 'request_id': 'req-fced2939-7200-4677-8caf-2ac999ad4210'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:23.543782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fddd1645636b456d8308a7a4b26d0e6e', 'request_id': 'req-fced2939-7200-4677-8caf-2ac999ad4210'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:23.575089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] The token cache is being invalidate because project fddd1645636b456d8308a7a4b26d0e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:20:23.575705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:23.575816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fced2939-7200-4677-8caf-2ac999ad4210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:20:23.577239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 590/1187] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:20:23 2022] DELETE /identity/v3/projects/fddd1645636b456d8308a7a4b26d0e6e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:20:33.025061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=62 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:20:33.573082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:21:53.355852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2884a-2285-4e61-ac2b-27982d1b7324 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:53.356485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2884a-2285-4e61-ac2b-27982d1b7324 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:53.356485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2884a-2285-4e61-ac2b-27982d1b7324 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:53.357774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 598/1188] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:21:53.362756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be776cd-9fc8-43f5-820d-0c180b7171f3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:53.363158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be776cd-9fc8-43f5-820d-0c180b7171f3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:53.363333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be776cd-9fc8-43f5-820d-0c180b7171f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:53.443078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9be776cd-9fc8-43f5-820d-0c180b7171f3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:21:53.444310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9be776cd-9fc8-43f5-820d-0c180b7171f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:21:53.546583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 591/1189] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:21:53.701400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2884a-2285-4e61-ac2b-27982d1b7324 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:21:53.704055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-68c2884a-2285-4e61-ac2b-27982d1b7324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:21:53.820563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2884a-2285-4e61-ac2b-27982d1b7324 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:21:53.825945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1fe42-a1b4-4852-b2bb-e07a1f3e4d08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:21:53.826552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1fe42-a1b4-4852-b2bb-e07a1f3e4d08 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:53.826831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1fe42-a1b4-4852-b2bb-e07a1f3e4d08 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:53.827653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1fe42-a1b4-4852-b2bb-e07a1f3e4d08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:53.835546 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1fe42-a1b4-4852-b2bb-e07a1f3e4d08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:21:53.837141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1fe42-a1b4-4852-b2bb-e07a1f3e4d08 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:21:53.846367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 599/1190] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:21:54.921261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61889-cc58-414d-89a8-e4055dffd1ab None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:54.921862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61889-cc58-414d-89a8-e4055dffd1ab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:54.921862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61889-cc58-414d-89a8-e4055dffd1ab None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:54.923148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 592/1191] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:21:54.927904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e273e81-9672-4ea7-bcc5-988c2b29ea9d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:54.928348 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e273e81-9672-4ea7-bcc5-988c2b29ea9d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:54.928491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e273e81-9672-4ea7-bcc5-988c2b29ea9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:54.948193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6e273e81-9672-4ea7-bcc5-988c2b29ea9d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:21:54.949423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6e273e81-9672-4ea7-bcc5-988c2b29ea9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:21:54.957377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 600/1192] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:21:54.998389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b89d2-c9c1-4074-b0e1-4ee5c9cabfb7 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:54.998788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b89d2-c9c1-4074-b0e1-4ee5c9cabfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:54.999677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b89d2-c9c1-4074-b0e1-4ee5c9cabfb7 None None] PATH_INFO: `/v3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:55.001040 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 593/1193] 10.222.0.45 () {54 vars in 925 bytes} [Thu Oct 27 13:21:54 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:21:55.007246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175ae6-361f-4ee6-841b-e091b62ff5ad None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:55.007531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175ae6-361f-4ee6-841b-e091b62ff5ad None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:55.007800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db175ae6-361f-4ee6-841b-e091b62ff5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:55.070012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-db175ae6-361f-4ee6-841b-e091b62ff5ad None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:21:55.071189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-db175ae6-361f-4ee6-841b-e091b62ff5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:21:55.111495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b89d2-c9c1-4074-b0e1-4ee5c9cabfb7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:21:55.122622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:21:55.126969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b89d2-c9c1-4074-b0e1-4ee5c9cabfb7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:21:55.132550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed41f3-ef43-4dd7-9910-b8288665291b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:21:55.133145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed41f3-ef43-4dd7-9910-b8288665291b service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:55.133417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed41f3-ef43-4dd7-9910-b8288665291b service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:55.133685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed41f3-ef43-4dd7-9910-b8288665291b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:55.142015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed41f3-ef43-4dd7-9910-b8288665291b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:21:55.143192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 601/1194] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Oct 27 13:21:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:21:55.143665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed41f3-ef43-4dd7-9910-b8288665291b service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:21:55.153436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 594/1195] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:21:58.649884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e407b-e150-43bc-9389-a2d82d6695f1 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:58.650347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e407b-e150-43bc-9389-a2d82d6695f1 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:58.650444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e407b-e150-43bc-9389-a2d82d6695f1 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:58.651772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 602/1196] 10.222.0.48 () {54 vars in 961 bytes} [Thu Oct 27 13:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:21:58.657256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16169ef0-e605-4fe0-b8a5-4158e69a3cb9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:58.657672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16169ef0-e605-4fe0-b8a5-4158e69a3cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:58.657787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16169ef0-e605-4fe0-b8a5-4158e69a3cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:58.677177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-16169ef0-e605-4fe0-b8a5-4158e69a3cb9 None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:21:58.678363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-16169ef0-e605-4fe0-b8a5-4158e69a3cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:21:58.686731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 595/1197] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Oct 27 13:21:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:21:58.694147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e407b-e150-43bc-9389-a2d82d6695f1 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:21:58.709416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e407b-e150-43bc-9389-a2d82d6695f1 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:21:58.714431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08421a51-c3e0-4df2-98a3-48e60a34f30e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:21:58.715021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08421a51-c3e0-4df2-98a3-48e60a34f30e service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:21:58.715292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08421a51-c3e0-4df2-98a3-48e60a34f30e service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:21:58.715559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08421a51-c3e0-4df2-98a3-48e60a34f30e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:21:58.725112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08421a51-c3e0-4df2-98a3-48e60a34f30e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:21:58.726667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08421a51-c3e0-4df2-98a3-48e60a34f30e service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:21:58.737555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 603/1198] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:21:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:22:06.600919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e854a-d050-42f8-b04e-52194f47cdb8 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:06.601461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e854a-d050-42f8-b04e-52194f47cdb8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:06.601461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e854a-d050-42f8-b04e-52194f47cdb8 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:06.602786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 596/1199] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:22:06.607595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700e6d-03e5-4b78-a4c1-c8f47484fc6b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:06.608027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700e6d-03e5-4b78-a4c1-c8f47484fc6b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:06.608125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700e6d-03e5-4b78-a4c1-c8f47484fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:06.628723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7a700e6d-03e5-4b78-a4c1-c8f47484fc6b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:22:06.629899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7a700e6d-03e5-4b78-a4c1-c8f47484fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:06.637771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 604/1200] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:22:06.790561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e854a-d050-42f8-b04e-52194f47cdb8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:22:06.808010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e854a-d050-42f8-b04e-52194f47cdb8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:22:06.814285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76401d-17ed-4424-ba9c-5404c8eac434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:22:06.814929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76401d-17ed-4424-ba9c-5404c8eac434 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:06.815213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76401d-17ed-4424-ba9c-5404c8eac434 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:06.815484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76401d-17ed-4424-ba9c-5404c8eac434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:06.824586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76401d-17ed-4424-ba9c-5404c8eac434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:22:06.826160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76401d-17ed-4424-ba9c-5404c8eac434 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:22:06.840268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 597/1201] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:22:07.888420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a89603-e399-40b2-9ddf-21439f4e566f None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:07.889490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a89603-e399-40b2-9ddf-21439f4e566f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:07.890306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a89603-e399-40b2-9ddf-21439f4e566f None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:07.891648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 605/1202] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 27 13:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:22:07.896454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07620-0469-4a3b-8832-3afc5895d60c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:07.896889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07620-0469-4a3b-8832-3afc5895d60c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:07.897002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07620-0469-4a3b-8832-3afc5895d60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:07.916068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ccc07620-0469-4a3b-8832-3afc5895d60c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:22:07.917303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ccc07620-0469-4a3b-8832-3afc5895d60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:07.925236 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 598/1203] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 27 13:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:22:08.081519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a89603-e399-40b2-9ddf-21439f4e566f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:22:08.095677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a89603-e399-40b2-9ddf-21439f4e566f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:22:08.101131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc351d-816e-467f-a12c-6b13bbfe51a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:22:08.101719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc351d-816e-467f-a12c-6b13bbfe51a8 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:08.101996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc351d-816e-467f-a12c-6b13bbfe51a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:08.102266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc351d-816e-467f-a12c-6b13bbfe51a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:08.110126 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc351d-816e-467f-a12c-6b13bbfe51a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:22:08.111678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc351d-816e-467f-a12c-6b13bbfe51a8 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:22:08.120778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 606/1204] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:22:12.143980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8a6a-6d38-460b-a3cf-36d47394ea54 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:12.144527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8a6a-6d38-460b-a3cf-36d47394ea54 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:12.144527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8a6a-6d38-460b-a3cf-36d47394ea54 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:12.145899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 599/1205] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:22:12.150987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58449215-a205-4dca-bad3-a84e27805c78 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:12.151388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58449215-a205-4dca-bad3-a84e27805c78 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:12.151519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58449215-a205-4dca-bad3-a84e27805c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:12.170925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-58449215-a205-4dca-bad3-a84e27805c78 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:22:12.172159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-58449215-a205-4dca-bad3-a84e27805c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:12.179788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 607/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:22:12.466679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d8a6a-6d38-460b-a3cf-36d47394ea54 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:22:12.468437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3f1d8a6a-6d38-460b-a3cf-36d47394ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:12.501084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:22:12.545963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d8a6a-6d38-460b-a3cf-36d47394ea54 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:22:12.549681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cfdc0-d3fb-42bd-9718-73093fac9d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:22:12.550284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfdc0-d3fb-42bd-9718-73093fac9d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:12.550562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfdc0-d3fb-42bd-9718-73093fac9d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:12.550836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfdc0-d3fb-42bd-9718-73093fac9d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:12.559702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cfdc0-d3fb-42bd-9718-73093fac9d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:22:12.561663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cfdc0-d3fb-42bd-9718-73093fac9d80 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:22:12.571743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 600/1207] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:22:14.718794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c475e-7580-47e5-83ca-83a6c96ceb4b None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:14.719589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c475e-7580-47e5-83ca-83a6c96ceb4b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:14.719775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c475e-7580-47e5-83ca-83a6c96ceb4b None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:14.721159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 608/1208] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:22:14.726338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36470c-a478-4dea-bbae-52a7be4c20ad None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:14.726836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36470c-a478-4dea-bbae-52a7be4c20ad None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:14.726970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36470c-a478-4dea-bbae-52a7be4c20ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:14.746419 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1f36470c-a478-4dea-bbae-52a7be4c20ad None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:22:14.747610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1f36470c-a478-4dea-bbae-52a7be4c20ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:14.755272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 601/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:22:15.042425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c475e-7580-47e5-83ca-83a6c96ceb4b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:22:15.054562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c475e-7580-47e5-83ca-83a6c96ceb4b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:22:15.058063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5284fc4-d9b9-453c-96b5-a8dbdc56c7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:22:15.058653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5284fc4-d9b9-453c-96b5-a8dbdc56c7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:15.058923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5284fc4-d9b9-453c-96b5-a8dbdc56c7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:15.059211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5284fc4-d9b9-453c-96b5-a8dbdc56c7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:15.067891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5284fc4-d9b9-453c-96b5-a8dbdc56c7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:22:15.069336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5284fc4-d9b9-453c-96b5-a8dbdc56c7b1 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:22:15.078843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 609/1210] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:22:18.728445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cec06-1c2f-4cd4-9e62-c73ec7358a40 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:18.728996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cec06-1c2f-4cd4-9e62-c73ec7358a40 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:18.728996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cec06-1c2f-4cd4-9e62-c73ec7358a40 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:18.730272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 602/1211] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:22:18.735964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bae05-1424-4994-a250-1842f96c613a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:18.736433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bae05-1424-4994-a250-1842f96c613a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:18.736562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bae05-1424-4994-a250-1842f96c613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:18.756284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-526bae05-1424-4994-a250-1842f96c613a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:22:18.757468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-526bae05-1424-4994-a250-1842f96c613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:18.765896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 610/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:22:19.068310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cec06-1c2f-4cd4-9e62-c73ec7358a40 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:22:19.069977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d20cec06-1c2f-4cd4-9e62-c73ec7358a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:19.146398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cec06-1c2f-4cd4-9e62-c73ec7358a40 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:22:19.149910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad5c4d-3a16-4bec-b8d8-78e835fca55d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:22:19.150495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5c4d-3a16-4bec-b8d8-78e835fca55d service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:19.150767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5c4d-3a16-4bec-b8d8-78e835fca55d service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:19.151031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5c4d-3a16-4bec-b8d8-78e835fca55d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:19.159583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad5c4d-3a16-4bec-b8d8-78e835fca55d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:22:19.161010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad5c4d-3a16-4bec-b8d8-78e835fca55d service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:22:19.170021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 603/1213] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:22:21.283267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1cbd0-650b-4686-a9e2-dd11bcbe7240 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:21.283828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1cbd0-650b-4686-a9e2-dd11bcbe7240 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:21.283828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1cbd0-650b-4686-a9e2-dd11bcbe7240 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:21.285152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 611/1214] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 27 13:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:22:21.290260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ccc64-4f9c-41c5-b36e-3068bee74d39 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:21.290681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ccc64-4f9c-41c5-b36e-3068bee74d39 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:21.290807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ccc64-4f9c-41c5-b36e-3068bee74d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:21.310278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ed4ccc64-4f9c-41c5-b36e-3068bee74d39 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:22:21.312027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ed4ccc64-4f9c-41c5-b36e-3068bee74d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:21.319617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 604/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 27 13:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:22:21.617892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1cbd0-650b-4686-a9e2-dd11bcbe7240 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:22:21.619501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-78b1cbd0-650b-4686-a9e2-dd11bcbe7240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:22:21.632593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1cbd0-650b-4686-a9e2-dd11bcbe7240 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:22:21.635988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911ee5-c592-4c13-8d32-c3dd5dadcc48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:22:21.636845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911ee5-c592-4c13-8d32-c3dd5dadcc48 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:22:21.637114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911ee5-c592-4c13-8d32-c3dd5dadcc48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:22:21.637375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911ee5-c592-4c13-8d32-c3dd5dadcc48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:22:21.646489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911ee5-c592-4c13-8d32-c3dd5dadcc48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:22:21.647884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911ee5-c592-4c13-8d32-c3dd5dadcc48 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:22:21.656936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 612/1216] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:22:31.305484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=63 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:22:31.652888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=37 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:45.229855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdbe4f-acbf-45c4-89f1-04fe2675ee49 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.230536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdbe4f-acbf-45c4-89f1-04fe2675ee49 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.230536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdbe4f-acbf-45c4-89f1-04fe2675ee49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.255475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-76cdbe4f-acbf-45c4-89f1-04fe2675ee49 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.256715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-76cdbe4f-acbf-45c4-89f1-04fe2675ee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.268261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 605/1217] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.279168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002968-3377-4e95-b4e8-7d80b74f0708 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.279734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002968-3377-4e95-b4e8-7d80b74f0708 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.279734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002968-3377-4e95-b4e8-7d80b74f0708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.310132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-85002968-3377-4e95-b4e8-7d80b74f0708 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.311336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-85002968-3377-4e95-b4e8-7d80b74f0708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.323308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 613/1218] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.328716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdbe4f-acbf-45c4-89f1-04fe2675ee49 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.338882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cdbe4f-acbf-45c4-89f1-04fe2675ee49 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.351771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.352413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.352690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.352959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.353623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:45.353906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.355633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:45.374293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 606/1219] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:45.382211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002968-3377-4e95-b4e8-7d80b74f0708 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.395528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002968-3377-4e95-b4e8-7d80b74f0708 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.403868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0590e1-f22e-4cf5-9d51-e5622ad01e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.404508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0590e1-f22e-4cf5-9d51-e5622ad01e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.404789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0590e1-f22e-4cf5-9d51-e5622ad01e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.405173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0590e1-f22e-4cf5-9d51-e5622ad01e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.405833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0590e1-f22e-4cf5-9d51-e5622ad01e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.407868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0590e1-f22e-4cf5-9d51-e5622ad01e70 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:45.469111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 614/1220] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:45.473590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.489957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c90f6-06d2-4062-bfeb-9600f4d539e9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.590220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b832b-a761-4130-88be-3bdf148339cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.591737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b832b-a761-4130-88be-3bdf148339cb admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.592032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b832b-a761-4130-88be-3bdf148339cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.592348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b832b-a761-4130-88be-3bdf148339cb admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.593006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b832b-a761-4130-88be-3bdf148339cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.595110 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b832b-a761-4130-88be-3bdf148339cb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:45.607504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab585e2-1db1-424b-9e36-3b8b9e5799a8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.607856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab585e2-1db1-424b-9e36-3b8b9e5799a8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.608043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab585e2-1db1-424b-9e36-3b8b9e5799a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.643576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 607/1221] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:45.644912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1facb-2984-4aaa-9f9a-a012946cc8e7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.645197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1facb-2984-4aaa-9f9a-a012946cc8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.645471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1facb-2984-4aaa-9f9a-a012946cc8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.651492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-fab585e2-1db1-424b-9e36-3b8b9e5799a8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.653023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fab585e2-1db1-424b-9e36-3b8b9e5799a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.671317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 615/1222] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:23:45.672171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab585e2-1db1-424b-9e36-3b8b9e5799a8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.684172 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-82f1facb-2984-4aaa-9f9a-a012946cc8e7 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.685527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-82f1facb-2984-4aaa-9f9a-a012946cc8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.689333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab585e2-1db1-424b-9e36-3b8b9e5799a8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.699548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b969a-0a7f-4dfe-954e-97b6d895ecc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.700704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b969a-0a7f-4dfe-954e-97b6d895ecc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.700990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b969a-0a7f-4dfe-954e-97b6d895ecc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.701305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b969a-0a7f-4dfe-954e-97b6d895ecc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.701954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b969a-0a7f-4dfe-954e-97b6d895ecc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.703543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b969a-0a7f-4dfe-954e-97b6d895ecc0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:45.707077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 608/1223] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.708351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b34acd-abd3-4f13-9f4a-7bd4a55032ee None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.708632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b34acd-abd3-4f13-9f4a-7bd4a55032ee None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.708900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b34acd-abd3-4f13-9f4a-7bd4a55032ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.726919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 616/1224] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:45.728452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10137a-5302-4cb7-8d98-00e89e0eba70 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.728745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10137a-5302-4cb7-8d98-00e89e0eba70 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.729016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10137a-5302-4cb7-8d98-00e89e0eba70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.755442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-41b34acd-abd3-4f13-9f4a-7bd4a55032ee None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.756654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-41b34acd-abd3-4f13-9f4a-7bd4a55032ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.775125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0b10137a-5302-4cb7-8d98-00e89e0eba70 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.776340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0b10137a-5302-4cb7-8d98-00e89e0eba70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.780313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 609/1225] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.781313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfa531-dc96-4f24-9ade-cd2350327de7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.781593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfa531-dc96-4f24-9ade-cd2350327de7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.781861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfa531-dc96-4f24-9ade-cd2350327de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.802855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 617/1226] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.803718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10137a-5302-4cb7-8d98-00e89e0eba70 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.817644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10137a-5302-4cb7-8d98-00e89e0eba70 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.819049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c1cfa531-dc96-4f24-9ade-cd2350327de7 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.820257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c1cfa531-dc96-4f24-9ade-cd2350327de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.828729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575cda3-5ce6-41ab-a07d-0f398ddf0ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.829351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575cda3-5ce6-41ab-a07d-0f398ddf0ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.829628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575cda3-5ce6-41ab-a07d-0f398ddf0ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.829894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575cda3-5ce6-41ab-a07d-0f398ddf0ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.830532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575cda3-5ce6-41ab-a07d-0f398ddf0ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.832121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575cda3-5ce6-41ab-a07d-0f398ddf0ddf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:45.842428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 610/1227] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.843695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f8f97-e226-49f9-a69a-760536ebe3a1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.843974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f8f97-e226-49f9-a69a-760536ebe3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.844255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f8f97-e226-49f9-a69a-760536ebe3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.852937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 618/1228] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:45.854282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145feb77-05c7-42ea-9eac-e13939bbaa56 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.854565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145feb77-05c7-42ea-9eac-e13939bbaa56 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.854840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145feb77-05c7-42ea-9eac-e13939bbaa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.867685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-139f8f97-e226-49f9-a69a-760536ebe3a1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.868894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-139f8f97-e226-49f9-a69a-760536ebe3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.875463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-145feb77-05c7-42ea-9eac-e13939bbaa56 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.876655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-145feb77-05c7-42ea-9eac-e13939bbaa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.876949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 611/1229] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.877712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f8f97-e226-49f9-a69a-760536ebe3a1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.887504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f8f97-e226-49f9-a69a-760536ebe3a1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.888200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 619/1230] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.889629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f50637-933d-4b16-ac9b-e2230b2acf0d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.889906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f50637-933d-4b16-ac9b-e2230b2acf0d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.890168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f50637-933d-4b16-ac9b-e2230b2acf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.893005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.893802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.894079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.894345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.894985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:45.895282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.896927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:45.909235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 612/1231] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:45.909583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-72f50637-933d-4b16-ac9b-e2230b2acf0d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.910301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.910820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-72f50637-933d-4b16-ac9b-e2230b2acf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.919670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 620/1232] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.920980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730334ff-f815-4fb5-83e0-0c1c93ff84ad None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.921379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730334ff-f815-4fb5-83e0-0c1c93ff84ad None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.921729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730334ff-f815-4fb5-83e0-0c1c93ff84ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.929407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f7eb8-1d7c-4118-b905-5a0f5424ff27 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.934241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.935109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.935382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.935661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.936326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:45.936626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.938209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:45.940842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-730334ff-f815-4fb5-83e0-0c1c93ff84ad None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.942023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-730334ff-f815-4fb5-83e0-0c1c93ff84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.949774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 621/1233] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.950963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 613/1234] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:45.951552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147a73e-15a5-4c02-b4db-e225fe2938dc None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.951765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147a73e-15a5-4c02-b4db-e225fe2938dc None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.952032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147a73e-15a5-4c02-b4db-e225fe2938dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.957153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:45.968032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484dcf9-b0da-49a8-92ac-6880707963e5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:45.972649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:45.973285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.973556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.973821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] PATH_INFO: `/v3/projects/cecd3a5ba83e47119462cb9166e9a80a/users/24289ae31c444a03b59ca5322f5abff2/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.974976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1147a73e-15a5-4c02-b4db-e225fe2938dc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:45.976400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1147a73e-15a5-4c02-b4db-e225fe2938dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:45.985450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 622/1235] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:45.986623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11912d6d-a14e-4e04-a794-3a1d60218084 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:45.986888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11912d6d-a14e-4e04-a794-3a1d60218084 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:45.987159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11912d6d-a14e-4e04-a794-3a1d60218084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:45.996150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cecd3a5ba83e47119462cb9166e9a80a, user_id=24289ae31c444a03b59ca5322f5abff2, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:45.998218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.006225 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-11912d6d-a14e-4e04-a794-3a1d60218084 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:46.007394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-11912d6d-a14e-4e04-a794-3a1d60218084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:46.012487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 614/1236] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:45 2022] PUT /identity/v3/projects/cecd3a5ba83e47119462cb9166e9a80a/users/24289ae31c444a03b59ca5322f5abff2/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:46.013862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.066915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:46.069857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 623/1237] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:46.070697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11912d6d-a14e-4e04-a794-3a1d60218084 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.079355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8c64d-0682-4ec7-b095-fc6d825f7672 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.080205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11912d6d-a14e-4e04-a794-3a1d60218084 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.084045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b607676-e36d-44a4-9ef4-146fc0d1bc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.084681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607676-e36d-44a4-9ef4-146fc0d1bc0e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.084832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.085006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607676-e36d-44a4-9ef4-146fc0d1bc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.085251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607676-e36d-44a4-9ef4-146fc0d1bc0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.085367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.085641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.085889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b607676-e36d-44a4-9ef4-146fc0d1bc0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:46.085990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.086186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b607676-e36d-44a4-9ef4-146fc0d1bc0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.086543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.087772 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b607676-e36d-44a4-9ef4-146fc0d1bc0e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.088426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.107455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 615/1238] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:46.107455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e01b23-fe26-4c17-b94a-0ad7432955f9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.107455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e01b23-fe26-4c17-b94a-0ad7432955f9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.107455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e01b23-fe26-4c17-b94a-0ad7432955f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.121123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 624/1239] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.121918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.123008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b0e01b23-fe26-4c17-b94a-0ad7432955f9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:46.124332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b0e01b23-fe26-4c17-b94a-0ad7432955f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:46.164756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:46.186556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2af30-0a22-40dc-a6e4-8bcea49047ea admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.187087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 616/1240] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:46.188391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490bcc7-dcb1-44f3-9b2e-f53ad0059124 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.188670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490bcc7-dcb1-44f3-9b2e-f53ad0059124 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.188940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490bcc7-dcb1-44f3-9b2e-f53ad0059124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.194021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83a7ab-6305-4fda-ab6e-50e3ae9a3dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.194634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a7ab-6305-4fda-ab6e-50e3ae9a3dff admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.194911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a7ab-6305-4fda-ab6e-50e3ae9a3dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.195178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a7ab-6305-4fda-ab6e-50e3ae9a3dff admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.195802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83a7ab-6305-4fda-ab6e-50e3ae9a3dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.197733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83a7ab-6305-4fda-ab6e-50e3ae9a3dff admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.208449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4490bcc7-dcb1-44f3-9b2e-f53ad0059124 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:46.209621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4490bcc7-dcb1-44f3-9b2e-f53ad0059124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:46.217682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 617/1241] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:46.219069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a650717-8fd7-40fe-8c4a-4c0c47ccf041 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.219345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a650717-8fd7-40fe-8c4a-4c0c47ccf041 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.219610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a650717-8fd7-40fe-8c4a-4c0c47ccf041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.227705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 625/1242] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.229204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5873-97d0-473a-8923-904327474e39 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.229488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5873-97d0-473a-8923-904327474e39 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.229754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5873-97d0-473a-8923-904327474e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.240948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2a650717-8fd7-40fe-8c4a-4c0c47ccf041 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:46.242308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2a650717-8fd7-40fe-8c4a-4c0c47ccf041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:46.249313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7baf5873-97d0-473a-8923-904327474e39 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:46.250471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7baf5873-97d0-473a-8923-904327474e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:46.302222 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 618/1243] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:46.302989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a650717-8fd7-40fe-8c4a-4c0c47ccf041 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.310083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 626/1244] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:46.313152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf5873-97d0-473a-8923-904327474e39 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.314340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a650717-8fd7-40fe-8c4a-4c0c47ccf041 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.319038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.319649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.319926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.320262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.320891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:46.321194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.322773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.322898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf5873-97d0-473a-8923-904327474e39 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.328708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.329308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.329583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.329849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.330470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.332384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.335257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 619/1245] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:46.336035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.346139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f9b0-00ce-4215-bfc7-0c93298f4b8f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.351704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.352365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.352698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.352966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.353590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.355755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.363907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 627/1246] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.363907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.381292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 620/1247] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.381292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.389330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084165f5-18c4-4487-884f-3638d5c2d5bb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.425260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bc8ab-bd31-4d26-8f1f-8845bbc7dabd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.429822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.430452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.430729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.431017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.431625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:46.431920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.432967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:23:46.432967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:23:46.432967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:23:46.433841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.438167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:46.446443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 628/1248] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:46.447338 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.450336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.450952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.451245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.451509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.452165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:46.452499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.454105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.457423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6fb5d-ad8e-4509-9cc2-9cd0e5eb6ca5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.462244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.462902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.463184 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.463452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.464099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.465132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:23:46.465132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:23:46.465132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:23:46.466350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.468105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 621/1249] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:46.469019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.478660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221947f4-9ba6-4591-9096-0195b7a567e8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.484726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.485352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.485623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.485890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.486518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:46.486814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.488477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.490497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 629/1250] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.491314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.504306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 622/1251] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:46.505215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.513741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:46.515304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261195-8f5d-43a8-bb26-6306edca5345 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.520426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.521029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.521304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.521567 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.522187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.523180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:23:46.523180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:23:46.523180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:23:46.524458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.550380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99f3c0-38bb-4e94-8d5c-507273f3dbbf admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.551554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 623/1252] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.552574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.603313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.603919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.604347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.604622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.613622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b398a81-d00b-42c0-a29c-8a166009571e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.614074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.615536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.620518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.621207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.621557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.621826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.624840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.626172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 630/1253] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:46.626797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.627260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.636185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98e93d-df59-4233-96b9-df082b2e2d4a service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.642586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.643190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.643465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.643737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.644408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.645986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.650072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 624/1254] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.650743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.660646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e52f95-52fe-4045-8c64-2814e7e3d892 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.664124 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 631/1255] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:46.664124 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.665472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.666081 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.666359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.666633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.667293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.669759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.673979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd09ed-ad52-455a-82f6-d94ba7656e54 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.679352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.679947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.680221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.680575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.681630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.684263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.702753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 625/1256] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.703809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.717180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 632/1257] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.718660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.778191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661a931-a0a2-414b-8dcc-4e452c33315f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.795406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093eed8-5b94-4290-b6ce-26eef2516cd4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.800188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.801092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.801092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.801410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.801983 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.802878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:46.803860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.832859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 633/1258] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.833850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.848159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.848895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.849238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.849754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.850807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.853034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.874199 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 626/1259] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:46.875491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.877484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:46.887776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e766b-bb78-44fb-83bd-bc4ecf5b6603 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.912799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7a055-c927-4371-87bf-25538b4f77fd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.918522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.919730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.920047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.920426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.920982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.923012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.948445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 634/1260] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:46.949395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.950061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.950402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.950663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.951125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.951301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.952907 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.962760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ed5aa-b282-4b0d-b71e-d632ea913a64 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.963451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 627/1261] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:46.964419 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.971707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.972356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.972638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.972906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.973554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.975140 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:46.985677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b5e41-268b-42e1-ac52-0fa59291ed51 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:46.989230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 635/1262] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:46.989230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:46.993227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:46.993990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:46.994340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:46.994863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:46.995551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:46.998182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.001757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2775-0518-4350-9e5b-b4d2622ffdd0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.007675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.008584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.008871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.009268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.009910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.012456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.033998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 628/1263] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/users => generated 294 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:47.037728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.045448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 636/1264] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:46 2022] POST /identity/v3/users => generated 302 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:47.045448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.052282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e52800-be58-4daa-ba67-53d8b384479c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.054307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e77356-2076-4a88-8181-39003972336d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.058460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.059258 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.059703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.059850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.060262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.060465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.060742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.061061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.061397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.061703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.063550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.063860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.080052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 637/1265] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.080052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.089853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3657f92-c3af-4b3b-bdb9-943031aae714 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.095540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 629/1266] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:47 2022] POST /identity/v3/users => generated 286 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:47.096752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.101014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.101707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.102145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.102494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.103298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.105059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.109843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076de003-7a5f-4fff-8a32-99646f31fca3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.115724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.116416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.116683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.119714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.119714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.119714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.120419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 638/1267] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.121471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.130818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 630/1268] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.131511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99eeba-93f0-4b82-83c5-9c3ea64aea0b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.131972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.137562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.138439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.138712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.139002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] PATH_INFO: `/v3/projects/a58f7c57ef7f4a54999d16d765d80cc2/users/97d696ea0031417d8a1a0b8471c26579/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.144513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f169a-4d7b-43e6-8657-ea86b2c7605a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.149600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.150354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.150636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.150917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.151529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.153135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.155261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58f7c57ef7f4a54999d16d765d80cc2, user_id=97d696ea0031417d8a1a0b8471c26579, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.157988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.164589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 631/1269] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.165486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.174887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 639/1270] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:47 2022] PUT /identity/v3/projects/a58f7c57ef7f4a54999d16d765d80cc2/users/97d696ea0031417d8a1a0b8471c26579/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:47.178738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670bb91-676b-4091-bb77-ac0b8c7bba2f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.178856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.239829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.240833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.241146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.241731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.242225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.243819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.244446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e376a-4534-4d34-87e2-a85f246a5326 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.245497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:47.250455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.251123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.251393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.251656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] PATH_INFO: `/v3/projects/5d1a967f9b62401ba95d65a2747e3ec0/users/8f0bb483c548429295c02558520c6898/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.259653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 632/1271] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.261168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.274486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6a2a4-6f17-4365-8eb7-4e9922b0e8dd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.280295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.280482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1a967f9b62401ba95d65a2747e3ec0, user_id=8f0bb483c548429295c02558520c6898, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.281120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.281588 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.281908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.282886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.283527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.284776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.300263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 633/1272] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.300263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.305124 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 640/1273] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:47 2022] PUT /identity/v3/projects/5d1a967f9b62401ba95d65a2747e3ec0/users/8f0bb483c548429295c02558520c6898/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:47.305994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.309747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d0f14-f474-46f8-8236-bdf0d5359826 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.352867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:47.386327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.386937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.387391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.387518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.388692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.390427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.391798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a898a4-25b8-4dfe-9df4-b4ada721c074 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.398612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.398952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.399333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.399706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] PATH_INFO: `/v3/projects/dd8f76f411bf4316860b48a8955a73fa/users/770c33a2b0d64b6197d5085b07a9032f/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.402603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 634/1274] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.405319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.417302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249833-deb7-4978-a2b4-879a3a6758aa admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.422460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8f76f411bf4316860b48a8955a73fa, user_id=770c33a2b0d64b6197d5085b07a9032f, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.423049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ba405-1b1a-4f0d-a30a-39db8e506012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.423920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ba405-1b1a-4f0d-a30a-39db8e506012 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.424194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ba405-1b1a-4f0d-a30a-39db8e506012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.424523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ba405-1b1a-4f0d-a30a-39db8e506012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.425184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ba405-1b1a-4f0d-a30a-39db8e506012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.426877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.427052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ba405-1b1a-4f0d-a30a-39db8e506012 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.440373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 635/1275] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.441858 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc482202-5e0e-43b7-bf3e-8e4e3c99efdc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.442139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc482202-5e0e-43b7-bf3e-8e4e3c99efdc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.442402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc482202-5e0e-43b7-bf3e-8e4e3c99efdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.444155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 641/1276] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:47 2022] PUT /identity/v3/projects/dd8f76f411bf4316860b48a8955a73fa/users/770c33a2b0d64b6197d5085b07a9032f/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:47.449427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.480478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bc482202-5e0e-43b7-bf3e-8e4e3c99efdc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:47.482729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bc482202-5e0e-43b7-bf3e-8e4e3c99efdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:47.536042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5701a6-97b7-401e-b0e4-5582343ae443 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.540987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.541686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.542034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.542374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.543138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.545856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.558520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 642/1277] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.560170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.574112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 636/1278] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:47.574279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a26b4-e551-458a-b571-eff1b34de9d7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.574380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491fc5a-bd13-4bfc-b475-04594cc9c461 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.574380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491fc5a-bd13-4bfc-b475-04594cc9c461 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.574380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491fc5a-bd13-4bfc-b475-04594cc9c461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.581480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.583738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.583738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.584054 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.584840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.586576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.599489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 643/1279] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:47.600293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.602132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a491fc5a-bd13-4bfc-b475-04594cc9c461 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:47.603691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a491fc5a-bd13-4bfc-b475-04594cc9c461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:47.615003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122929f-eee4-4f6b-ab99-a6307a570ba7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.620466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beca126-335c-4f65-a7c3-0d238c3b1de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.621221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beca126-335c-4f65-a7c3-0d238c3b1de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.621570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beca126-335c-4f65-a7c3-0d238c3b1de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.625471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beca126-335c-4f65-a7c3-0d238c3b1de0 admin admin] PATH_INFO: `/v3/projects/f6200925c5054329a030ccb01d03fd6c/users/054ae31acea4491fa25e7b6620b22bad/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.626436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 637/1280] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:47.627525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491fc5a-bd13-4bfc-b475-04594cc9c461 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.641704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491fc5a-bd13-4bfc-b475-04594cc9c461 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.647908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.648631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.648909 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.649174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] PATH_INFO: `/v3/projects/a14a3cf9df9f4034a8a0109fd3e33816/users/e77228bb97fd40dfa311088f28864637/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.650781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beca126-335c-4f65-a7c3-0d238c3b1de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6200925c5054329a030ccb01d03fd6c, user_id=054ae31acea4491fa25e7b6620b22bad, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.654518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beca126-335c-4f65-a7c3-0d238c3b1de0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.672360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 644/1281] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:47 2022] PUT /identity/v3/projects/f6200925c5054329a030ccb01d03fd6c/users/054ae31acea4491fa25e7b6620b22bad/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:47.674425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59640d69-e71d-4c5f-8148-eeea63469c6a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.675479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59640d69-e71d-4c5f-8148-eeea63469c6a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.675759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59640d69-e71d-4c5f-8148-eeea63469c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.682043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14a3cf9df9f4034a8a0109fd3e33816, user_id=e77228bb97fd40dfa311088f28864637, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.685982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.703027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 638/1282] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:47 2022] PUT /identity/v3/projects/a14a3cf9df9f4034a8a0109fd3e33816/users/e77228bb97fd40dfa311088f28864637/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:47.704106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.719392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-59640d69-e71d-4c5f-8148-eeea63469c6a None None] MFA Rules not processed for user `770c33a2b0d64b6197d5085b07a9032f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:47.721453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-59640d69-e71d-4c5f-8148-eeea63469c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:47.760185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:47.789610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd15a5-d47c-4ef8-8af4-2b2a71f49015 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.798818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.800334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.800718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.801123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] PATH_INFO: `/v3/projects/181858f4783e4af280faee987f399b24/users/606da71a4db847e9bccb2da923231425/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.807522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 645/1283] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:47.808730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59640d69-e71d-4c5f-8148-eeea63469c6a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.813782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:47.821298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=181858f4783e4af280faee987f399b24, user_id=606da71a4db847e9bccb2da923231425, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.824116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.847140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 639/1284] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:47 2022] PUT /identity/v3/projects/181858f4783e4af280faee987f399b24/users/606da71a4db847e9bccb2da923231425/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:47.848252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.877048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59640d69-e71d-4c5f-8148-eeea63469c6a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.937963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.938582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.938953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.939230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.944170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266eb04b-bbf3-44b5-b85b-bc5fdd06ccc4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.949100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.951100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.952646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d17bc-dd6d-4b7f-bf1e-e2ea9e4ff84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.953551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d17bc-dd6d-4b7f-bf1e-e2ea9e4ff84e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.954062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d17bc-dd6d-4b7f-bf1e-e2ea9e4ff84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.954556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d17bc-dd6d-4b7f-bf1e-e2ea9e4ff84e admin admin] PATH_INFO: `/v3/projects/ba958741e9c24c729a5cff811019511f/users/2e7ebd94fa1043fc9df1d6b58f69dc38/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.961607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 646/1285] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:47.962547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:47.971674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8207a4f-9fb8-47f0-8315-9c4c7baf7efd service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:47.975708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06870a6f-d2e0-4b79-b42f-8daf9b169163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:47.976349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06870a6f-d2e0-4b79-b42f-8daf9b169163 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:47.976686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06870a6f-d2e0-4b79-b42f-8daf9b169163 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:47.976957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06870a6f-d2e0-4b79-b42f-8daf9b169163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:47.983012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d17bc-dd6d-4b7f-bf1e-e2ea9e4ff84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba958741e9c24c729a5cff811019511f, user_id=2e7ebd94fa1043fc9df1d6b58f69dc38, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.987841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06870a6f-d2e0-4b79-b42f-8daf9b169163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:47.989170 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d17bc-dd6d-4b7f-bf1e-e2ea9e4ff84e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:47.989542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06870a6f-d2e0-4b79-b42f-8daf9b169163 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.009416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 640/1286] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:47 2022] PUT /identity/v3/projects/ba958741e9c24c729a5cff811019511f/users/2e7ebd94fa1043fc9df1d6b58f69dc38/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:48.012074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1236-c16b-42ee-80ec-fff0bca07778 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.013086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1236-c16b-42ee-80ec-fff0bca07778 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.013601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1236-c16b-42ee-80ec-fff0bca07778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.022144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 647/1287] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:48.026464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bfa91-b02c-4d5f-9149-894bb0bbeea8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.026464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bfa91-b02c-4d5f-9149-894bb0bbeea8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.026650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bfa91-b02c-4d5f-9149-894bb0bbeea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.055669 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3f2d1236-c16b-42ee-80ec-fff0bca07778 None None] MFA Rules not processed for user `054ae31acea4491fa25e7b6620b22bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:48.056985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3f2d1236-c16b-42ee-80ec-fff0bca07778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:48.070182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1f8bfa91-b02c-4d5f-9149-894bb0bbeea8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:48.074000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1f8bfa91-b02c-4d5f-9149-894bb0bbeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:48.157201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 641/1288] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:48.158960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d1236-c16b-42ee-80ec-fff0bca07778 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.167235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 648/1289] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:48.168751 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce91f2e-47aa-44cb-abc7-0db0d46df64b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.169063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce91f2e-47aa-44cb-abc7-0db0d46df64b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.169629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce91f2e-47aa-44cb-abc7-0db0d46df64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.192805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d1236-c16b-42ee-80ec-fff0bca07778 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.193935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3ce91f2e-47aa-44cb-abc7-0db0d46df64b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:48.195295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3ce91f2e-47aa-44cb-abc7-0db0d46df64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:48.206971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.209182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.209883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.210145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 649/1290] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:48.211231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.211762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbc3b-0a1d-4583-8676-d292f676dcfd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.212106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbc3b-0a1d-4583-8676-d292f676dcfd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.213045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbc3b-0a1d-4583-8676-d292f676dcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.213760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.218027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.249190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-539cbc3b-0a1d-4583-8676-d292f676dcfd None None] MFA Rules not processed for user `2e7ebd94fa1043fc9df1d6b58f69dc38`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:48.250488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-539cbc3b-0a1d-4583-8676-d292f676dcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:48.271953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 642/1291] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:48.274090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.276433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:48.294390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:48.307189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:48.361509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 650/1292] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:48.363881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e73b02-4764-4b7c-8109-5e4ef6648a60 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.364491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e73b02-4764-4b7c-8109-5e4ef6648a60 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.364910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e73b02-4764-4b7c-8109-5e4ef6648a60 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.369685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 651/1293] 10.222.0.48 () {54 vars in 961 bytes} [Thu Oct 27 13:23:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:23:48.370886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e73b02-4764-4b7c-8109-5e4ef6648a60 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.419698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15662-c7e9-46c4-9c91-8a7de4d01189 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.424162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.424880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.425209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.425482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.483818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e73b02-4764-4b7c-8109-5e4ef6648a60 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.488428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.489098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.489377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.489701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.532840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.535717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.551407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 643/1294] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:48.552708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.565497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4e4d2-b901-4471-a078-a35b4a031421 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.571479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42ecaf-3970-4838-a76b-80dd6c676a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.572473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42ecaf-3970-4838-a76b-80dd6c676a82 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.572899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42ecaf-3970-4838-a76b-80dd6c676a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.573310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42ecaf-3970-4838-a76b-80dd6c676a82 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.574373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42ecaf-3970-4838-a76b-80dd6c676a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.577226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.577632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42ecaf-3970-4838-a76b-80dd6c676a82 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.578667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.590810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 652/1295] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:48.591736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.606055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e0465-ec68-4d46-b6de-0a0c409cded2 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.672291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.673243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 644/1296] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/users => generated 288 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:48.673475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.673475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.673475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.673475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.673764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4442a26-8c5a-43b8-afb1-74a2ea1966d5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.673764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4442a26-8c5a-43b8-afb1-74a2ea1966d5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.673764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4442a26-8c5a-43b8-afb1-74a2ea1966d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.673965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.714745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 653/1297] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:48.715337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.747827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a4442a26-8c5a-43b8-afb1-74a2ea1966d5 None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:48.752018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a4442a26-8c5a-43b8-afb1-74a2ea1966d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:48.843840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246304c-725d-4742-867e-3e71066f18a6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.851474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 645/1298] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:48.852617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4442a26-8c5a-43b8-afb1-74a2ea1966d5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.854934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.855863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.856151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.857309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.857969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.859799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.861955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:48.865873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4442a26-8c5a-43b8-afb1-74a2ea1966d5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.875114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.876021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.876511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.876901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.877661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 654/1299] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:48.877786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.880547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.881701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.899168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01fe9e-97f7-46fa-93b1-ae3b3770ebcc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.905496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.906104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.906382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.906655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.912411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 646/1300] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:48 2022] POST /identity/v3/users => generated 291 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:48.912411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:48.926901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb4780-e509-4860-8a92-281f0bf97af4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:48.935696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:48.937597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:48.938130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:48.938690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] PATH_INFO: `/v3/projects/a8b0c712b06a4b19a196af7f8d02298b/users/ba77440dd21c4f11b6d503db38dedc59/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:48.942191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:48.963090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b0c712b06a4b19a196af7f8d02298b, user_id=ba77440dd21c4f11b6d503db38dedc59, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:48.968448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:48.992046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 647/1301] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:48 2022] PUT /identity/v3/projects/a8b0c712b06a4b19a196af7f8d02298b/users/ba77440dd21c4f11b6d503db38dedc59/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:48.995684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.020036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.023031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.058706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 655/1302] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:49.061249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.082630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d39ac-1d03-4de5-a7d2-e6b011ffe618 service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.089864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.090895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.091280 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.091657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.092967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.095979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2074b6-468c-4c83-9ac4-5384356d59d9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.097651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.103736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.104475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.104870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.105234 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.106085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.109268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.120307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 656/1303] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:49.123767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 648/1304] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:49.125101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.128253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.148882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ef9c8-5580-4156-8a1c-25b19e902ace admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.153071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb4f0-194d-4875-b438-dc073fe08535 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.160242 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0fc8e-5824-4902-ac27-021b38c7fa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.161490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0fc8e-5824-4902-ac27-021b38c7fa3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.163246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0fc8e-5824-4902-ac27-021b38c7fa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.163576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0fc8e-5824-4902-ac27-021b38c7fa3a admin admin] PATH_INFO: `/v3/projects/a8b0c712b06a4b19a196af7f8d02298b/users/ba77440dd21c4f11b6d503db38dedc59/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.164426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.166129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.166411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.167225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] PATH_INFO: `/v3/projects/dc8b28be83964891930feb91a3f62b6e/users/bf5909a23fd74aa0982319c2715289d5/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.168796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0fc8e-5824-4902-ac27-021b38c7fa3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b0c712b06a4b19a196af7f8d02298b, user_id=ba77440dd21c4f11b6d503db38dedc59, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.171796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0fc8e-5824-4902-ac27-021b38c7fa3a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.206445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 657/1305] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:49 2022] PUT /identity/v3/projects/a8b0c712b06a4b19a196af7f8d02298b/users/ba77440dd21c4f11b6d503db38dedc59/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:49.207937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932739f2-b69b-41c1-b6c9-879df4f0f391 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.208224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932739f2-b69b-41c1-b6c9-879df4f0f391 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.209031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932739f2-b69b-41c1-b6c9-879df4f0f391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.212908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8b28be83964891930feb91a3f62b6e, user_id=bf5909a23fd74aa0982319c2715289d5, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.216020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.246635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 649/1306] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:49 2022] PUT /identity/v3/projects/dc8b28be83964891930feb91a3f62b6e/users/bf5909a23fd74aa0982319c2715289d5/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:49.251740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.286267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-932739f2-b69b-41c1-b6c9-879df4f0f391 None None] MFA Rules not processed for user `ba77440dd21c4f11b6d503db38dedc59`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:49.288115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-932739f2-b69b-41c1-b6c9-879df4f0f391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:49.400492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57e30a-2448-4a48-814c-9a48c0ca82cd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.414547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.415173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.415451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.415726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.416486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.419836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.468699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 650/1307] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:23:49.472764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.479381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 658/1308] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:49.492769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932739f2-b69b-41c1-b6c9-879df4f0f391 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.503566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe558a-d884-4492-b8a0-7d212c55fe83 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.526820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3bb35-1d31-4b3c-9d88-dfa2d456caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.527695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3bb35-1d31-4b3c-9d88-dfa2d456caf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.527827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3bb35-1d31-4b3c-9d88-dfa2d456caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.528037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3bb35-1d31-4b3c-9d88-dfa2d456caf2 admin admin] PATH_INFO: `/v3/projects/dc8b28be83964891930feb91a3f62b6e/users/bf5909a23fd74aa0982319c2715289d5/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.535985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3bb35-1d31-4b3c-9d88-dfa2d456caf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8b28be83964891930feb91a3f62b6e, user_id=bf5909a23fd74aa0982319c2715289d5, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.540393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3bb35-1d31-4b3c-9d88-dfa2d456caf2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.568528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 651/1309] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:49 2022] PUT /identity/v3/projects/dc8b28be83964891930feb91a3f62b6e/users/bf5909a23fd74aa0982319c2715289d5/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:23:49.576937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3fa6-2ad7-4def-81e2-a5d785ae4ea3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.577225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3fa6-2ad7-4def-81e2-a5d785ae4ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.577917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3fa6-2ad7-4def-81e2-a5d785ae4ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.629688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932739f2-b69b-41c1-b6c9-879df4f0f391 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.643366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-142f3fa6-2ad7-4def-81e2-a5d785ae4ea3 None None] MFA Rules not processed for user `bf5909a23fd74aa0982319c2715289d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:49.646104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-142f3fa6-2ad7-4def-81e2-a5d785ae4ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:49.649896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:49.726267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:49.772683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.775548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.775831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.787737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.841555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 652/1310] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:49.846699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3fa6-2ad7-4def-81e2-a5d785ae4ea3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:49.860974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3fa6-2ad7-4def-81e2-a5d785ae4ea3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:49.866368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b86933-bf1b-40cc-b4b2-a1192ab1bdd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:49.867320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b86933-bf1b-40cc-b4b2-a1192ab1bdd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:49.867781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b86933-bf1b-40cc-b4b2-a1192ab1bdd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:49.868413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b86933-bf1b-40cc-b4b2-a1192ab1bdd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:49.881797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b86933-bf1b-40cc-b4b2-a1192ab1bdd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.883909 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b86933-bf1b-40cc-b4b2-a1192ab1bdd9 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:49.912559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 653/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:49.983092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:49.991864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:50.007697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 659/1312] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:23:50.070954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be119a60-2df3-4094-9439-eeb264087fe2 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:50.071109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be119a60-2df3-4094-9439-eeb264087fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:50.071381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be119a60-2df3-4094-9439-eeb264087fe2 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:50.073106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 654/1313] 10.222.0.48 () {54 vars in 961 bytes} [Thu Oct 27 13:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:23:50.080043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:50.089657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ddafc-5d92-4453-9600-097ecbb50d01 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:50.098289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fd409-76cf-4847-b356-94645be8d828 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:50.098598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fd409-76cf-4847-b356-94645be8d828 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:50.098924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fd409-76cf-4847-b356-94645be8d828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:50.100328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:50.103063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:50.103364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:50.104468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:50.139533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-610fd409-76cf-4847-b356-94645be8d828 None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:50.143031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-610fd409-76cf-4847-b356-94645be8d828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:50.238174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:50.240994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:50.272733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 660/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:23:50.276145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:50.304928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e283c-e945-4463-a825-fac4b0a389c9 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:50.319893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89144301-2068-4c20-a163-a48b56a4adc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:50.320654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144301-2068-4c20-a163-a48b56a4adc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:50.320998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144301-2068-4c20-a163-a48b56a4adc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:50.321331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144301-2068-4c20-a163-a48b56a4adc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:50.321474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 655/1315] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Oct 27 13:23:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:50.342126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ff79-31a4-4ca8-b340-5987920f93fe None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:50.342648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ff79-31a4-4ca8-b340-5987920f93fe None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:50.342923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ff79-31a4-4ca8-b340-5987920f93fe None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:50.344832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 656/1316] 10.222.0.48 () {54 vars in 912 bytes} [Thu Oct 27 13:23:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:23:50.353110 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd736c8-249a-4493-b017-24a131a15bcd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:50.353364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd736c8-249a-4493-b017-24a131a15bcd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:50.353632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd736c8-249a-4493-b017-24a131a15bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:50.382684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7fd736c8-249a-4493-b017-24a131a15bcd None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:50.387125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7fd736c8-249a-4493-b017-24a131a15bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:50.425527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 657/1317] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:23:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:23:50.467795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89144301-2068-4c20-a163-a48b56a4adc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:50.469358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89144301-2068-4c20-a163-a48b56a4adc6 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:50.473324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:50.493915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 661/1318] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:50.919939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd736c8-249a-4493-b017-24a131a15bcd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:50.940186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd736c8-249a-4493-b017-24a131a15bcd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:50.952041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b2bad-e487-4496-a2b6-57dfc0008aa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:50.952695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2bad-e487-4496-a2b6-57dfc0008aa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:50.953073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2bad-e487-4496-a2b6-57dfc0008aa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:50.953253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2bad-e487-4496-a2b6-57dfc0008aa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:50.984640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b2bad-e487-4496-a2b6-57dfc0008aa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:50.986112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b2bad-e487-4496-a2b6-57dfc0008aa0 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:51.037251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 658/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:51.415947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba5a9c-653e-4850-a417-eeeeefb0ac8c None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:51.416455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba5a9c-653e-4850-a417-eeeeefb0ac8c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:51.416885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba5a9c-653e-4850-a417-eeeeefb0ac8c None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:51.419390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 662/1320] 10.222.0.48 () {54 vars in 912 bytes} [Thu Oct 27 13:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:23:51.425321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34187-df13-43ef-acea-0387cea737d1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:51.425817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34187-df13-43ef-acea-0387cea737d1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:51.426667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d34187-df13-43ef-acea-0387cea737d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:51.462645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a3d34187-df13-43ef-acea-0387cea737d1 None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:51.464159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a3d34187-df13-43ef-acea-0387cea737d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:51.484942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 659/1321] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:23:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:51.505759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba5a9c-653e-4850-a417-eeeeefb0ac8c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:51.523400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba5a9c-653e-4850-a417-eeeeefb0ac8c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:51.533251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e84e5-ede0-43b4-a260-2c2cfa70279c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:51.534308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e84e5-ede0-43b4-a260-2c2cfa70279c service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:51.534981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e84e5-ede0-43b4-a260-2c2cfa70279c service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:51.536185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e84e5-ede0-43b4-a260-2c2cfa70279c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:51.554151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e84e5-ede0-43b4-a260-2c2cfa70279c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:51.555957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e84e5-ede0-43b4-a260-2c2cfa70279c service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:51.570669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 663/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:51.643354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d34187-df13-43ef-acea-0387cea737d1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:51.795379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d34187-df13-43ef-acea-0387cea737d1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:51.801407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:51.802075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:51.802357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:51.802693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] PATH_INFO: `/v3/users/054ae31acea4491fa25e7b6620b22bad` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:51.804277 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054ae31acea4491fa25e7b6620b22bad)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:51.805373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: WARNING py.warnings [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:23:51.805373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: warnings.warn(msg) Oct 27 13:23:51.805373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: Oct 27 13:23:51.806579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:51.831319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618126-1fc0-430e-bec3-c6463d984444 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:51.831601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618126-1fc0-430e-bec3-c6463d984444 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:51.831872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618126-1fc0-430e-bec3-c6463d984444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:51.859392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:51.893810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4f618126-1fc0-430e-bec3-c6463d984444 None None] MFA Rules not processed for user `606da71a4db847e9bccb2da923231425`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:51.895097 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4f618126-1fc0-430e-bec3-c6463d984444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:51.946461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054ae31acea4491fa25e7b6620b22bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:51.990759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054ae31acea4491fa25e7b6620b22bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:51.991660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054ae31acea4491fa25e7b6620b22bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:52.011328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054ae31acea4491fa25e7b6620b22bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:52.024018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 664/1323] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:51 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:23:52.033990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f618126-1fc0-430e-bec3-c6463d984444 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:52.034185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-3ee2c824-fc2d-4a2b-b78f-57b7bd441af6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054ae31acea4491fa25e7b6620b22bad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:52.041650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 660/1324] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:23:51 2022] DELETE /identity/v3/users/054ae31acea4491fa25e7b6620b22bad => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:52.045632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.045926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.046206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.046635 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f618126-1fc0-430e-bec3-c6463d984444 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:52.086065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:52.087413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:52.190380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:52.191050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.191558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.191900 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.192669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:52.204090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:52.283397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 661/1325] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:52.295120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 665/1326] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:52 2022] POST /identity/v3/projects => generated 403 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:23:52.299349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:52.300819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:52.329724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898986db-9379-496a-8c64-c1de9c5562d8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:52.342771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:52.345231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:52.461878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:52.462519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.462773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.463038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.463675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:52.465690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:52.518758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 662/1327] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:52 2022] POST /identity/v3/users => generated 308 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:52.523106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:52.534623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee015893-fbf0-4962-8fb1-1dfb1ffaf9db admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:52.540889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:52.541788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.541925 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64133e-8d23-401b-a539-6db19524da73 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:52.542476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.542476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.555343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:52.555977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.556269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.556576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.557193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:52.558557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:52.558854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:52.561590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:52.579880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 666/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:52.581417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 663/1329] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:23:52.585597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:52.601353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:52.622920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea667e-6f33-46e5-98cd-3904dc2a9959 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:52.639734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:52.640715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.641896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.643415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] PATH_INFO: `/v3/projects/b9dc2acf6232478297944b4b777c2828/users/ac9c99cb85fc4ea8965e71f0ec0a2b99/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.698155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dc2acf6232478297944b4b777c2828, user_id=ac9c99cb85fc4ea8965e71f0ec0a2b99, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:52.705316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:52.738474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 667/1330] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:52 2022] PUT /identity/v3/projects/b9dc2acf6232478297944b4b777c2828/users/ac9c99cb85fc4ea8965e71f0ec0a2b99/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:52.743822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:52.760931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eee665-82a6-4ab6-b6f5-f57ede674b8a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:52.791335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:52.910767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:52.911410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.911692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.911963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.914371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:52.916154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:52.943804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 664/1331] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:52.945343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:52.962441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066b116-f108-4796-b718-1dee84db366b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:52.974601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf70bd-2d3c-4153-8220-eba0dfc554a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:52.978088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf70bd-2d3c-4153-8220-eba0dfc554a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:52.978382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf70bd-2d3c-4153-8220-eba0dfc554a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:52.979693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf70bd-2d3c-4153-8220-eba0dfc554a0 admin admin] PATH_INFO: `/v3/projects/b9dc2acf6232478297944b4b777c2828/users/ac9c99cb85fc4ea8965e71f0ec0a2b99/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:52.991195 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf70bd-2d3c-4153-8220-eba0dfc554a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dc2acf6232478297944b4b777c2828, user_id=ac9c99cb85fc4ea8965e71f0ec0a2b99, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:53.002300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf70bd-2d3c-4153-8220-eba0dfc554a0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:53.044490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 668/1332] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:52 2022] PUT /identity/v3/projects/b9dc2acf6232478297944b4b777c2828/users/ac9c99cb85fc4ea8965e71f0ec0a2b99/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:54.289056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:54.296027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:54.335200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef12d5f-b3ee-42d1-9427-d296b9b2272c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:54.547232 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:54.548267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:54.548416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:54.549032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] PATH_INFO: `/v3/projects/f6200925c5054329a030ccb01d03fd6c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:54.550461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6200925c5054329a030ccb01d03fd6c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:54.558056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:54.733182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6200925c5054329a030ccb01d03fd6c', 'request_id': 'req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:54.749356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6200925c5054329a030ccb01d03fd6c', 'request_id': 'req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:54.758493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6200925c5054329a030ccb01d03fd6c', 'request_id': 'req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:54.787462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] The token cache is being invalidate because project f6200925c5054329a030ccb01d03fd6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:23:54.789534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:54.792333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-01cb3dfb-fb24-4c59-8e55-69f04ecc4e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:23:54.794284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 665/1333] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:23:54 2022] DELETE /identity/v3/projects/f6200925c5054329a030ccb01d03fd6c => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:54.801000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe99b4-794a-433b-ac88-d2ca914c2829 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:54.801542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe99b4-794a-433b-ac88-d2ca914c2829 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:54.801624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe99b4-794a-433b-ac88-d2ca914c2829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:54.866905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e2fe99b4-794a-433b-ac88-d2ca914c2829 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:54.868142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e2fe99b4-794a-433b-ac88-d2ca914c2829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:55.016679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6fba5-5feb-4f5f-8884-0f7c174bfd93 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.017246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6fba5-5feb-4f5f-8884-0f7c174bfd93 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.018216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6fba5-5feb-4f5f-8884-0f7c174bfd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.019287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 669/1334] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:55.065402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-15e6fba5-5feb-4f5f-8884-0f7c174bfd93 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:55.066985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-15e6fba5-5feb-4f5f-8884-0f7c174bfd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:55.088749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 666/1335] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:55.093008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe99b4-794a-433b-ac88-d2ca914c2829 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:55.108174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe99b4-794a-433b-ac88-d2ca914c2829 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:55.122613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:55.123603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.123890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.124161 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.124852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:23:55.125149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:55.127797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:55.163524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 670/1336] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:23:55.164218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6fba5-5feb-4f5f-8884-0f7c174bfd93 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:55.179173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6fba5-5feb-4f5f-8884-0f7c174bfd93 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:55.190709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:55.191587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.192063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.192566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.193415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:55.195518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:55.258886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 667/1337] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:55 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:55.265797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:55.287259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2a36-5a9a-4a68-b69c-07baee533de3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:55.455588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:55.458956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.459323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.461056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.464621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:55.472106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:55.549707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 671/1338] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:55 2022] POST /identity/v3/users => generated 289 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:55.554274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:55.592177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad9ed-fbd9-41d1-a0a0-c9fde951d4ea admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:55.619512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03216a-c218-4e2a-a0b8-990f1c46ecff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:55.619883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03216a-c218-4e2a-a0b8-990f1c46ecff admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.620411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03216a-c218-4e2a-a0b8-990f1c46ecff admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.620822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03216a-c218-4e2a-a0b8-990f1c46ecff admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.623327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03216a-c218-4e2a-a0b8-990f1c46ecff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:55.636762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03216a-c218-4e2a-a0b8-990f1c46ecff admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:55.666008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:55.691882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:55.692641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 668/1339] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:55.708827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45099e3-050f-4531-a5df-f31ba315ef7a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.709135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45099e3-050f-4531-a5df-f31ba315ef7a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.709381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45099e3-050f-4531-a5df-f31ba315ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.712844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94f28c-4cb3-4fee-a72c-e3f4f28d0af2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:55.734317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:55.735194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.735821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.736369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.741170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:55.743357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:55.783715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 672/1340] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:55.783931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d45099e3-050f-4531-a5df-f31ba315ef7a None None] MFA Rules not processed for user `ac9c99cb85fc4ea8965e71f0ec0a2b99`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:55.789148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:55.793260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d45099e3-050f-4531-a5df-f31ba315ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:55.814643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4daf4-0be9-4000-82c6-ecaa206af67c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:55.834774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895b0d-63f5-4ca6-a549-aa12a6cb3beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:55.835502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895b0d-63f5-4ca6-a549-aa12a6cb3beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.836264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895b0d-63f5-4ca6-a549-aa12a6cb3beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.838241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895b0d-63f5-4ca6-a549-aa12a6cb3beb admin admin] PATH_INFO: `/v3/projects/3bae2cce8069489fa0d283e4011982b4/users/3d012ea9fb264ba2b74df8dc803db381/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.905532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57895b0d-63f5-4ca6-a549-aa12a6cb3beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bae2cce8069489fa0d283e4011982b4, user_id=3d012ea9fb264ba2b74df8dc803db381, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:55.915218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57895b0d-63f5-4ca6-a549-aa12a6cb3beb admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:55.952461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 673/1341] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:55 2022] PUT /identity/v3/projects/3bae2cce8069489fa0d283e4011982b4/users/3d012ea9fb264ba2b74df8dc803db381/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:55.962277 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 669/1342] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:55 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:23:55.969326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bf953-b01c-4d3c-a98e-4f5b4cb7b58d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:55.969581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bf953-b01c-4d3c-a98e-4f5b4cb7b58d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:55.969855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bf953-b01c-4d3c-a98e-4f5b4cb7b58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:55.997175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:56.024164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0d6bf953-b01c-4d3c-a98e-4f5b4cb7b58d None None] MFA Rules not processed for user `3d012ea9fb264ba2b74df8dc803db381`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:56.027644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0d6bf953-b01c-4d3c-a98e-4f5b4cb7b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:56.120274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 674/1343] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:56.123689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45099e3-050f-4531-a5df-f31ba315ef7a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:56.136308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45099e3-050f-4531-a5df-f31ba315ef7a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:56.235832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4e35e-64ce-41f0-85fd-5511db7564ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:56.236917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e35e-64ce-41f0-85fd-5511db7564ed admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:56.237348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e35e-64ce-41f0-85fd-5511db7564ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:56.237792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4e35e-64ce-41f0-85fd-5511db7564ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:56.239209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e35e-64ce-41f0-85fd-5511db7564ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:56.241996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4e35e-64ce-41f0-85fd-5511db7564ed admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:56.291620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 670/1344] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:23:56 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:23:56.295620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bf953-b01c-4d3c-a98e-4f5b4cb7b58d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:56.308867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bf953-b01c-4d3c-a98e-4f5b4cb7b58d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:56.386846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfd54-7c06-41d6-b760-8a36f91e7617 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:56.387729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfd54-7c06-41d6-b760-8a36f91e7617 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:56.388015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfd54-7c06-41d6-b760-8a36f91e7617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:56.471135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:56.472411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:56.474668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:56.475190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:56.476067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:56.479212 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:56.480965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-77cdfd54-7c06-41d6-b760-8a36f91e7617 None None] MFA Rules not processed for user `24289ae31c444a03b59ca5322f5abff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:56.482402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-77cdfd54-7c06-41d6-b760-8a36f91e7617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:56.538704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 675/1345] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:23:56 2022] POST /identity/v3/users => generated 295 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:23:56.542413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:56.579899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f93c5d-f720-4407-b496-c4cccd213ebf admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:56.593230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:56.598589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:56.599665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:56.600536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:56.601698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:56.605122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:56.631585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:56.642133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 671/1346] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:56.652866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:56.653186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:56.653416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:56.675027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 676/1347] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:23:56.682830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:56.709379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] MFA Rules not processed for user `24289ae31c444a03b59ca5322f5abff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:56.710246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c54f7b-2caa-4678-b85d-75d3f7ad764c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:56.710931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:56.717350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:56.718299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:56.718781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:56.727901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] PATH_INFO: `/v3/projects/f435f558891348c3be4cb79a8dbc528d/users/ceef3d13766d4ee197eb954cd2be3e38/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:56.732221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 672/1348] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:23:56.745592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:56.748274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:56.829598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f435f558891348c3be4cb79a8dbc528d, user_id=ceef3d13766d4ee197eb954cd2be3e38, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:56.833541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:56.863096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 677/1349] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:56 2022] PUT /identity/v3/projects/f435f558891348c3be4cb79a8dbc528d/users/ceef3d13766d4ee197eb954cd2be3e38/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:56.869115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:56.982222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:56.998765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f821834-5e35-40b2-96f1-cebd2bf1b9c2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:57.026239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d5077-c006-428c-9b50-48b97a394d44 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:57.038085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:57.039750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:57.040807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:57.041305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:57.045042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:57.050061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:57.072386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 678/1350] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:23:57.077793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:57.093332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1184ead-0ca3-484f-af70-a78059f85fd6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:57.142175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f122296-e0f5-45f7-99da-6657f6c92a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:57.142804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f122296-e0f5-45f7-99da-6657f6c92a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:57.145190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f122296-e0f5-45f7-99da-6657f6c92a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:57.145479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f122296-e0f5-45f7-99da-6657f6c92a0e admin admin] PATH_INFO: `/v3/projects/f435f558891348c3be4cb79a8dbc528d/users/ceef3d13766d4ee197eb954cd2be3e38/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:57.157518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f122296-e0f5-45f7-99da-6657f6c92a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f435f558891348c3be4cb79a8dbc528d, user_id=ceef3d13766d4ee197eb954cd2be3e38, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:57.167025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f122296-e0f5-45f7-99da-6657f6c92a0e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:57.176450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:57.177315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:57.177754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:57.178166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:57.205533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 679/1351] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:23:57 2022] PUT /identity/v3/projects/f435f558891348c3be4cb79a8dbc528d/users/ceef3d13766d4ee197eb954cd2be3e38/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:23:57.216508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee522-dff4-4cc6-9843-9374c144e75e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:57.217137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee522-dff4-4cc6-9843-9374c144e75e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:57.217642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ee522-dff4-4cc6-9843-9374c144e75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:57.297535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4b6ee522-dff4-4cc6-9843-9374c144e75e None None] MFA Rules not processed for user `ceef3d13766d4ee197eb954cd2be3e38`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:57.299658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4b6ee522-dff4-4cc6-9843-9374c144e75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:57.318583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:57.321505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:57.362148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 673/1352] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:57.463100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 680/1353] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:23:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:57.474975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:57.476738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:57.588846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:23:57.654283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97027e26-aec5-4ea9-a7f8-6f7bdf8b5ecb service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:57.773020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd844b-2944-4a8e-916e-64cd64af79ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:57.773823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd844b-2944-4a8e-916e-64cd64af79ec service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:57.774195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd844b-2944-4a8e-916e-64cd64af79ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:57.774582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd844b-2944-4a8e-916e-64cd64af79ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:57.958620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd844b-2944-4a8e-916e-64cd64af79ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:57.960449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd844b-2944-4a8e-916e-64cd64af79ec service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:57.990158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 674/1354] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:58.832202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df7f90-b482-41ee-a3a3-1776132567ea None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:58.832728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df7f90-b482-41ee-a3a3-1776132567ea None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:58.832812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df7f90-b482-41ee-a3a3-1776132567ea None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:58.834355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 681/1355] 10.222.0.48 () {54 vars in 912 bytes} [Thu Oct 27 13:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:23:58.848323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:58.849127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:58.849127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:58.969337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] MFA Rules not processed for user `b7cba7b6049e41488f5b2f7c61a0500a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:58.975701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:59.006885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2349c-7006-45d1-a338-47fa617fdbbd None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:59.007548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2349c-7006-45d1-a338-47fa617fdbbd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:59.007548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2349c-7006-45d1-a338-47fa617fdbbd None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:59.008839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 682/1356] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:23:59.019685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:59.020162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:59.020467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:59.076849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:23:59.078139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:59.187678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 675/1357] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:23:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:23:59.264214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 683/1358] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:23:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:23:59.279560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:59.281434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:59.297776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9afe0-5b60-4d0e-9cd8-3fb61da201f4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:59.323595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:59.325678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:59.326523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:59.326796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:59.359759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:59.361241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:59.377636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 676/1359] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:23:59.520221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:23:59.522076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:23:59.600412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2f33a-dbbe-450d-a749-7985d8badf66 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:23:59.618286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:23:59.618997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:23:59.619279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:23:59.619543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:23:59.640082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:23:59.642426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:23:59.671248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 684/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:23:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:01.048150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:01.051136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:01.125247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:01.279673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8d7bf-9858-4bde-8827-abb31bc1e83f service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:01.299625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:01.301788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:01.305140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:01.309546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] PATH_INFO: `/v3/users/606da71a4db847e9bccb2da923231425` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:01.312602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] RBAC: Authorizing `identity:delete_user(user_id=606da71a4db847e9bccb2da923231425)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:01.314596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:01.372217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:01.393201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dd09c-d6f3-4d14-a9bb-de30c1f1ab43 service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:01.399706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2926c66-631b-4dfa-bf00-219ad268741f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:01.401126 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926c66-631b-4dfa-bf00-219ad268741f service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:01.401763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926c66-631b-4dfa-bf00-219ad268741f service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:01.403453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926c66-631b-4dfa-bf00-219ad268741f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:01.405612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d2926c66-631b-4dfa-bf00-219ad268741f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:01.422762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '606da71a4db847e9bccb2da923231425'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:01.432726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2926c66-631b-4dfa-bf00-219ad268741f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:01.438494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2926c66-631b-4dfa-bf00-219ad268741f service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:01.455540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '606da71a4db847e9bccb2da923231425'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:01.457057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '606da71a4db847e9bccb2da923231425'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:01.464376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 685/1361] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:01.467812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '606da71a4db847e9bccb2da923231425'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:01.481319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '606da71a4db847e9bccb2da923231425'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:01.490647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 677/1362] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:01 2022] DELETE /identity/v3/users/606da71a4db847e9bccb2da923231425 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:02.816479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:02.818589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:02.825500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:02.870297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:02.872825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:02.942942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] MFA Rules not processed for user `8f0bb483c548429295c02558520c6898`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:02.947551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:03.007340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:03.109407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813edefa-17a7-440c-beaf-cde9d804656e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:03.123538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:03.124417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:03.124802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:03.125187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] PATH_INFO: `/v3/projects/181858f4783e4af280faee987f399b24` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:03.128944 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] RBAC: Authorizing `identity:delete_project(project_id=181858f4783e4af280faee987f399b24)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:03.132837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:03.285179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:03.291650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 686/1363] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:03.296122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181858f4783e4af280faee987f399b24', 'request_id': 'req-0b81e3fb-e58e-4304-867a-6204aa4d439a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.331333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:03.333828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:03.338729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181858f4783e4af280faee987f399b24', 'request_id': 'req-0b81e3fb-e58e-4304-867a-6204aa4d439a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.351142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181858f4783e4af280faee987f399b24', 'request_id': 'req-0b81e3fb-e58e-4304-867a-6204aa4d439a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.389050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] The token cache is being invalidate because project 181858f4783e4af280faee987f399b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:03.391246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.393184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.397345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 678/1364] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:02 2022] DELETE /identity/v3/projects/181858f4783e4af280faee987f399b24 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:03.407803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:03.411041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:03.611532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d0f1-49c3-4a95-a1ef-0b867ac18e5e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:03.628961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e3fb-e58e-4304-867a-6204aa4d439a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:03.646044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:03.648649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:03.648940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:03.649286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] PATH_INFO: `/v3/users/ac9c99cb85fc4ea8965e71f0ec0a2b99` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:03.651965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9c99cb85fc4ea8965e71f0ec0a2b99)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:03.657617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:03.797362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9c99cb85fc4ea8965e71f0ec0a2b99'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.827283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9c99cb85fc4ea8965e71f0ec0a2b99'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.828889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9c99cb85fc4ea8965e71f0ec0a2b99'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.851613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9c99cb85fc4ea8965e71f0ec0a2b99'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.873733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9c99cb85fc4ea8965e71f0ec0a2b99'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:03.875765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbece4-8746-4bea-8e3a-c1add336c27b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:03.876979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbece4-8746-4bea-8e3a-c1add336c27b service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:03.877490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbece4-8746-4bea-8e3a-c1add336c27b service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:03.877909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbece4-8746-4bea-8e3a-c1add336c27b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:03.882201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d4cbece4-8746-4bea-8e3a-c1add336c27b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:03.885829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 679/1365] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:03 2022] DELETE /identity/v3/users/ac9c99cb85fc4ea8965e71f0ec0a2b99 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:04.070477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:04.082253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbece4-8746-4bea-8e3a-c1add336c27b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:04.086616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbece4-8746-4bea-8e3a-c1add336c27b service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:04.126348 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 687/1366] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:04.778697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:04.780320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:04.929433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:04.972814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa2fff-3a68-4f90-a469-0eef44bb7736 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:04.992069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:04.993222 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:04.993723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:04.995766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] PATH_INFO: `/v3/projects/b9dc2acf6232478297944b4b777c2828` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:04.995944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a210e8-38b6-495c-94ac-308b7a717902 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:04.997781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a210e8-38b6-495c-94ac-308b7a717902 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:04.998436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a210e8-38b6-495c-94ac-308b7a717902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.007468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9dc2acf6232478297944b4b777c2828)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:05.010825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:05.084562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d1a210e8-38b6-495c-94ac-308b7a717902 None None] MFA Rules not processed for user `97d696ea0031417d8a1a0b8471c26579`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:05.086638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d1a210e8-38b6-495c-94ac-308b7a717902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:05.157575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9dc2acf6232478297944b4b777c2828', 'request_id': 'req-7128ee01-073e-4c0b-9f12-1ba4d7389639'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:05.176437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9dc2acf6232478297944b4b777c2828', 'request_id': 'req-7128ee01-073e-4c0b-9f12-1ba4d7389639'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:05.185947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9dc2acf6232478297944b4b777c2828', 'request_id': 'req-7128ee01-073e-4c0b-9f12-1ba4d7389639'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:05.208908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] The token cache is being invalidate because project b9dc2acf6232478297944b4b777c2828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:05.210056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:05.211366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7128ee01-073e-4c0b-9f12-1ba4d7389639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:05.216279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 680/1367] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:04 2022] DELETE /identity/v3/projects/b9dc2acf6232478297944b4b777c2828 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:05.221377 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688756-84b2-4598-8847-98b6661d6469 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:05.221792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688756-84b2-4598-8847-98b6661d6469 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:05.222159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688756-84b2-4598-8847-98b6661d6469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.258133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-25688756-84b2-4598-8847-98b6661d6469 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:05.261140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-25688756-84b2-4598-8847-98b6661d6469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:05.350555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 688/1368] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:05.367013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e531f-fd57-4428-8599-8fd52296956a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:05.367314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e531f-fd57-4428-8599-8fd52296956a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:05.367586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e531f-fd57-4428-8599-8fd52296956a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.392798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-be7e531f-fd57-4428-8599-8fd52296956a None None] MFA Rules not processed for user `97d696ea0031417d8a1a0b8471c26579`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:05.394539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-be7e531f-fd57-4428-8599-8fd52296956a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:05.426214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 681/1369] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:05.435684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:05.435963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:05.436352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.529734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:05.531847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:05.534698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 689/1370] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:05.542867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd308a-df8f-45cc-b9f7-79fc69c70bea None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:05.543762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd308a-df8f-45cc-b9f7-79fc69c70bea None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:05.544131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd308a-df8f-45cc-b9f7-79fc69c70bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.565315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 682/1371] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:05.568753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:05.572471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:05.599575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-23fd308a-df8f-45cc-b9f7-79fc69c70bea None None] MFA Rules not processed for user `e77228bb97fd40dfa311088f28864637`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:05.602252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-23fd308a-df8f-45cc-b9f7-79fc69c70bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:05.731734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 690/1372] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:05.731734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd308a-df8f-45cc-b9f7-79fc69c70bea None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:05.741131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd308a-df8f-45cc-b9f7-79fc69c70bea None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:05.746265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:05.746975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:05.747352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:05.747694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.748483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:05.748848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:05.751080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:05.763661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d978a-7da2-40c7-a8d5-afe38ce4d528 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:05.768272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 691/1373] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:05.771698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:05.778171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:05.778953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:05.779325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:05.779692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.788366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:05.794754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b7873-bc34-4c93-b12c-0b82ff01d394 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:05.795219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:05.798257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:05.807822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:05.808982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:05.810002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:05.810002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:05.810149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:05.810745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:05.814206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:05.820560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 683/1374] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:05.856936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 692/1375] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:05 2022] POST /identity/v3/projects => generated 379 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:05.860379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:05.873191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a5ee7-9ad9-4de8-9f70-c1aa40bbe132 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:05.950246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:05.952986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:06.028683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:06.032337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:06.032620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:06.033307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:06.033957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:06.043985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:06.089750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 684/1376] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:05 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:06.094471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:06.120405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3b409-b15c-42f1-961e-b7cfc9b378c8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:06.129184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:06.130541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:06.132467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:06.132629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:06.134833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:06.137948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622f303-23db-4228-b24d-8972e6743cbc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:06.139333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:06.154346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af6388-b8bd-4cc3-85a0-0718222e30a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:06.155560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af6388-b8bd-4cc3-85a0-0718222e30a5 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:06.157553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af6388-b8bd-4cc3-85a0-0718222e30a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:06.157836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af6388-b8bd-4cc3-85a0-0718222e30a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:06.170177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 685/1377] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:06.171879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:06.195666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a59cc8-40d0-4ab4-a840-d6dedd37ddc3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:06.230852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:06.234750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:06.235190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:06.236390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:06.237680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:06.248696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:06.287187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 686/1378] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:06.288423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:06.317278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850592-2c1c-4e48-b3b5-7f5f29f83f2a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:06.326746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:06.327423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:06.327697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:06.327967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] PATH_INFO: `/v3/projects/4332bfb660d145f7bb81f02ed2c020ba/users/94fc181fc107418fbc0ef34d91017f3a/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:06.342557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af6388-b8bd-4cc3-85a0-0718222e30a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:06.344326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af6388-b8bd-4cc3-85a0-0718222e30a5 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:06.375255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4332bfb660d145f7bb81f02ed2c020ba, user_id=94fc181fc107418fbc0ef34d91017f3a, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:06.377104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 693/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:06.379008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:06.415053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 687/1380] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:06 2022] PUT /identity/v3/projects/4332bfb660d145f7bb81f02ed2c020ba/users/94fc181fc107418fbc0ef34d91017f3a/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:06.425352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef9cc-b137-4485-b510-17d3e1d0f5e9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:06.425637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef9cc-b137-4485-b510-17d3e1d0f5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:06.425909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef9cc-b137-4485-b510-17d3e1d0f5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:06.500166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b13ef9cc-b137-4485-b510-17d3e1d0f5e9 None None] MFA Rules not processed for user `94fc181fc107418fbc0ef34d91017f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:06.502737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b13ef9cc-b137-4485-b510-17d3e1d0f5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:06.638870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 694/1381] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:06.644323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:06.788436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0274-15ff-4fff-b268-065191ed396d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:06.812540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:06.813336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:06.813615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:06.813900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:06.814533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:06.816508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:06.865823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 688/1382] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:06 2022] POST /identity/v3/projects => generated 379 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:06.870231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef9cc-b137-4485-b510-17d3e1d0f5e9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:06.884683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef9cc-b137-4485-b510-17d3e1d0f5e9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:06.898612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:07.026173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:07.029905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:07.030783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:07.032677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:07.033620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:07.043383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:07.108737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 695/1383] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:06 2022] POST /identity/v3/users => generated 296 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:07.110180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:07.127204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fafb2-696f-44d9-b9b1-421b75d97641 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:07.139702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f30724-5fd9-4335-a524-6f45134e7a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:07.140748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f30724-5fd9-4335-a524-6f45134e7a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:07.141406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f30724-5fd9-4335-a524-6f45134e7a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:07.141910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f30724-5fd9-4335-a524-6f45134e7a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:07.144134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f30724-5fd9-4335-a524-6f45134e7a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:07.148985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f30724-5fd9-4335-a524-6f45134e7a26 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:07.178847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 689/1384] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:07.181760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:07.228076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8b90c-15c7-4d1f-af34-24bd539affe4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:07.240090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:07.241060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:07.241554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:07.242043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] PATH_INFO: `/v3/projects/70c11efb1eb44853b12de09e7cdb2fc7/users/fb32252da18943049ae90ca8fbab491b/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:07.257010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae3f32-68dd-4c24-9fe9-e77d7fe9a278 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:07.257329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae3f32-68dd-4c24-9fe9-e77d7fe9a278 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:07.258212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae3f32-68dd-4c24-9fe9-e77d7fe9a278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:07.274316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c11efb1eb44853b12de09e7cdb2fc7, user_id=fb32252da18943049ae90ca8fbab491b, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:07.280858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:07.312024 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 696/1385] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:07 2022] PUT /identity/v3/projects/70c11efb1eb44853b12de09e7cdb2fc7/users/fb32252da18943049ae90ca8fbab491b/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:07.314562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:07.318961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f3ae3f32-68dd-4c24-9fe9-e77d7fe9a278 None None] MFA Rules not processed for user `97d696ea0031417d8a1a0b8471c26579`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:07.335323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f3ae3f32-68dd-4c24-9fe9-e77d7fe9a278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:07.402011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:07.524780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa890e-b3cc-42ba-bdda-1144fbc5493a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:07.547395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:07.549363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:07.550215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:07.552217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:07.553799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:07.557730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:07.590855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 697/1386] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:07.591765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:07.597803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:07.662564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 690/1387] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:07.663475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae3f32-68dd-4c24-9fe9-e77d7fe9a278 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:07.689257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae3f32-68dd-4c24-9fe9-e77d7fe9a278 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:07.697571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539ca74-afde-493c-8a30-74d3756143ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:07.698462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ca74-afde-493c-8a30-74d3756143ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:07.698864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ca74-afde-493c-8a30-74d3756143ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:07.699252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539ca74-afde-493c-8a30-74d3756143ac admin admin] PATH_INFO: `/v3/projects/70c11efb1eb44853b12de09e7cdb2fc7/users/fb32252da18943049ae90ca8fbab491b/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:07.701505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539ca74-afde-493c-8a30-74d3756143ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c11efb1eb44853b12de09e7cdb2fc7, user_id=fb32252da18943049ae90ca8fbab491b, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:07.704364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539ca74-afde-493c-8a30-74d3756143ac admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:07.761880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 691/1388] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:07 2022] PUT /identity/v3/projects/70c11efb1eb44853b12de09e7cdb2fc7/users/fb32252da18943049ae90ca8fbab491b/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:07.763422 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5e48-e42f-47ab-8817-05480c4e5a7d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:07.764346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5e48-e42f-47ab-8817-05480c4e5a7d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:07.765335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5e48-e42f-47ab-8817-05480c4e5a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:07.850531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e2f1f-f29b-421c-86e9-bb678b484c10 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:07.864586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-de7b5e48-e42f-47ab-8817-05480c4e5a7d None None] MFA Rules not processed for user `fb32252da18943049ae90ca8fbab491b`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:07.867414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-de7b5e48-e42f-47ab-8817-05480c4e5a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:07.936993 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:08.013508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:08.014199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:08.014475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:08.014744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:08.019425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:08.202585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 692/1389] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:08.282107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086ac79-3d4d-4b08-8a1f-7aaa64fde36e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:08.282397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086ac79-3d4d-4b08-8a1f-7aaa64fde36e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:08.282732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086ac79-3d4d-4b08-8a1f-7aaa64fde36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:08.366556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:08.367928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:08.405516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 698/1390] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 814 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:08.421116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7086ac79-3d4d-4b08-8a1f-7aaa64fde36e None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:08.422385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7086ac79-3d4d-4b08-8a1f-7aaa64fde36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:08.528647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:08.530172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:08.567235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49b59-85c9-492b-af9f-993ec43d77bd service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:08.587776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:08.591385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:08.598824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:08.599220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:08.654414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:08.661214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:08.689373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 693/1391] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:24:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:08.709340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03228-f117-4833-bc72-d2238b0505e4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:08.709636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03228-f117-4833-bc72-d2238b0505e4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:08.713846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03228-f117-4833-bc72-d2238b0505e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:08.731977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 699/1392] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:08.808715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:08.840563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0cb03228-f117-4833-bc72-d2238b0505e4 None None] MFA Rules not processed for user `8f0bb483c548429295c02558520c6898`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:08.845256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0cb03228-f117-4833-bc72-d2238b0505e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:09.001221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:09.002809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:09.108275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 694/1393] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:09.249886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03228-f117-4833-bc72-d2238b0505e4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:09.279746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03228-f117-4833-bc72-d2238b0505e4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:09.291548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7167-c1ea-4c34-803d-d20766a5d074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:09.292689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7167-c1ea-4c34-803d-d20766a5d074 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:09.293462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7167-c1ea-4c34-803d-d20766a5d074 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:09.294381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7167-c1ea-4c34-803d-d20766a5d074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:09.309651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3960e3-40c4-4f57-9210-fb12c6e6d119 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:09.334824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48837d3c-fc2d-430f-a55d-cf8b8eeadf0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:09.335793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48837d3c-fc2d-430f-a55d-cf8b8eeadf0a service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:09.337669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48837d3c-fc2d-430f-a55d-cf8b8eeadf0a service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:09.338017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48837d3c-fc2d-430f-a55d-cf8b8eeadf0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:09.369110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48837d3c-fc2d-430f-a55d-cf8b8eeadf0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:09.377188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48837d3c-fc2d-430f-a55d-cf8b8eeadf0a service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:09.439024 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 700/1394] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:24:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 13:24:09.470138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7167-c1ea-4c34-803d-d20766a5d074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:09.485693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7167-c1ea-4c34-803d-d20766a5d074 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:09.540199 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 695/1395] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:10.591789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384f55-09f7-465b-9db8-bb8ca1c77d41 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:10.592374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384f55-09f7-465b-9db8-bb8ca1c77d41 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:10.592462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384f55-09f7-465b-9db8-bb8ca1c77d41 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:10.594342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 701/1396] 10.222.0.48 () {54 vars in 912 bytes} [Thu Oct 27 13:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:24:10.605910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:10.606387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:10.606657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:10.690083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] MFA Rules not processed for user `b7cba7b6049e41488f5b2f7c61a0500a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:10.691389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:10.811590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384f55-09f7-465b-9db8-bb8ca1c77d41 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:10.813541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-71384f55-09f7-465b-9db8-bb8ca1c77d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:10.835653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 696/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:24:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:10.857439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:10.861719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:10.884735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0697be-2e90-4125-93f1-12bf90b98c8f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:10.897195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d23c1-7a74-458c-8163-d922d64f38de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:10.898186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d23c1-7a74-458c-8163-d922d64f38de service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:10.898568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d23c1-7a74-458c-8163-d922d64f38de service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:10.898784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d23c1-7a74-458c-8163-d922d64f38de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:10.914289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d23c1-7a74-458c-8163-d922d64f38de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:10.916834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d23c1-7a74-458c-8163-d922d64f38de service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:10.918282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:10.944812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 697/1398] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:10.993287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384f55-09f7-465b-9db8-bb8ca1c77d41 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:11.005282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:11.007615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:11.008598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:11.008888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] PATH_INFO: `/v3/users/770c33a2b0d64b6197d5085b07a9032f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:11.016068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=770c33a2b0d64b6197d5085b07a9032f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:11.018580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:11.144541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770c33a2b0d64b6197d5085b07a9032f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:11.187527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770c33a2b0d64b6197d5085b07a9032f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:11.188500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770c33a2b0d64b6197d5085b07a9032f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:11.211952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770c33a2b0d64b6197d5085b07a9032f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:11.228894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770c33a2b0d64b6197d5085b07a9032f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:11.242318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:11.242598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:11.242872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:11.253375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 702/1399] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:10 2022] DELETE /identity/v3/users/770c33a2b0d64b6197d5085b07a9032f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:11.300323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:11.309338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:11.461525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:11.586008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 698/1400] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:11.711617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:11.713116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:12.087512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854ddd4-1fb4-4229-9eb4-6517b5928a5c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:12.092181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:12.092852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:12.093136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:12.093408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:12.130873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:12.161054 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:12.258172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 703/1401] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:13.340338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:13.342342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:13.356917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40666-ccba-4380-a3b3-3b2d2b7d3572 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:13.362281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:13.363119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:13.363541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:13.363942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] PATH_INFO: `/v3/projects/dd8f76f411bf4316860b48a8955a73fa` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:13.365301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8f76f411bf4316860b48a8955a73fa)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:13.367352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:13.452664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8f76f411bf4316860b48a8955a73fa', 'request_id': 'req-67128336-0bbe-4a68-bbed-85d6a2b203ad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.469356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8f76f411bf4316860b48a8955a73fa', 'request_id': 'req-67128336-0bbe-4a68-bbed-85d6a2b203ad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.484470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8f76f411bf4316860b48a8955a73fa', 'request_id': 'req-67128336-0bbe-4a68-bbed-85d6a2b203ad'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.511287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] The token cache is being invalidate because project dd8f76f411bf4316860b48a8955a73fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:13.512307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.513111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-67128336-0bbe-4a68-bbed-85d6a2b203ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.520157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:13.520771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 699/1402] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:13 2022] DELETE /identity/v3/projects/dd8f76f411bf4316860b48a8955a73fa => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:13.538294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c446d-cb3e-47cb-a23d-93aaf9558852 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:13.598766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dd605-3a74-4ef4-b173-52abb4916760 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:13.598971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dd605-3a74-4ef4-b173-52abb4916760 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:13.599241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dd605-3a74-4ef4-b173-52abb4916760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:13.656798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c96dd605-3a74-4ef4-b173-52abb4916760 None None] MFA Rules not processed for user `e77228bb97fd40dfa311088f28864637`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:13.658417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c96dd605-3a74-4ef4-b173-52abb4916760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:13.680604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:13.704402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:13.705282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:13.705619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:13.705951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] PATH_INFO: `/v3/users/ba77440dd21c4f11b6d503db38dedc59` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:13.714194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba77440dd21c4f11b6d503db38dedc59)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:13.717117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:13.834689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 700/1403] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:13.926074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba77440dd21c4f11b6d503db38dedc59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.958025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:13.958534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:13.958948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:13.963800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba77440dd21c4f11b6d503db38dedc59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.965302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba77440dd21c4f11b6d503db38dedc59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:13.990962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba77440dd21c4f11b6d503db38dedc59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:14.007912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] MFA Rules not processed for user `b7cba7b6049e41488f5b2f7c61a0500a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:14.011355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba77440dd21c4f11b6d503db38dedc59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:14.024045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:14.030517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 704/1404] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:13 2022] DELETE /identity/v3/users/ba77440dd21c4f11b6d503db38dedc59 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:14.211227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 701/1405] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Oct 27 13:24:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:14.809802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:14.812555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:14.845632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84790ca0-c068-4972-be87-4270b84b1ecc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:14.859540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5acfa-de8e-45d9-bb0b-465cde8890fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:14.862957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5acfa-de8e-45d9-bb0b-465cde8890fe service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:14.863731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5acfa-de8e-45d9-bb0b-465cde8890fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:14.866896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5acfa-de8e-45d9-bb0b-465cde8890fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:14.890216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5acfa-de8e-45d9-bb0b-465cde8890fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:14.894214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5acfa-de8e-45d9-bb0b-465cde8890fe service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:14.936481 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 705/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:16.420137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:16.423722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:16.474006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:16.523549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdabdf-0162-4dc2-9c05-b6ce86c14cfd None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:16.530925 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:16.532270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:16.532270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:16.532270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] PATH_INFO: `/v3/projects/a8b0c712b06a4b19a196af7f8d02298b` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:16.533871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b0c712b06a4b19a196af7f8d02298b)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:16.536126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:16.606767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b0c712b06a4b19a196af7f8d02298b', 'request_id': 'req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:16.624158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b0c712b06a4b19a196af7f8d02298b', 'request_id': 'req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:16.646717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b0c712b06a4b19a196af7f8d02298b', 'request_id': 'req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:16.662135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] The token cache is being invalidate because project a8b0c712b06a4b19a196af7f8d02298b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:16.662135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:16.664685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-002e6042-5e20-4a6b-bb3d-2a4e66c0ea0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:16.666717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 702/1407] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:16 2022] DELETE /identity/v3/projects/a8b0c712b06a4b19a196af7f8d02298b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:16.672782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88fab3-9217-41c4-bc96-5321e61f7742 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:16.673826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88fab3-9217-41c4-bc96-5321e61f7742 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:16.674632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88fab3-9217-41c4-bc96-5321e61f7742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:16.716496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-aa88fab3-9217-41c4-bc96-5321e61f7742 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:16.717729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aa88fab3-9217-41c4-bc96-5321e61f7742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:16.813319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 706/1408] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:16.815197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c48ef-4534-422d-974b-fde1db0a92c4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:16.815622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c48ef-4534-422d-974b-fde1db0a92c4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:16.816037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c48ef-4534-422d-974b-fde1db0a92c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:16.843815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e95c48ef-4534-422d-974b-fde1db0a92c4 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:16.846036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e95c48ef-4534-422d-974b-fde1db0a92c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:16.877758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 703/1409] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:16.879139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88fab3-9217-41c4-bc96-5321e61f7742 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:16.895266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88fab3-9217-41c4-bc96-5321e61f7742 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:16.901265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:16.902864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:16.903347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:16.903952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:16.905738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:16.906234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:16.908046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:16.922865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 707/1410] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:16.927060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c48ef-4534-422d-974b-fde1db0a92c4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:16.969035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c48ef-4534-422d-974b-fde1db0a92c4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:16.987861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:16.988683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:16.989688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:16.990217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:16.991938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:17.007709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:17.110613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 704/1411] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:16 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:17.111848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:17.126495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dd493-8bf3-464b-8260-657bb26d9ea5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:17.272270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:17.301785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:17.302851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.304434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.304917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.306664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:17.310990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:17.396046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:17.397158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 708/1412] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:17 2022] POST /identity/v3/users => generated 292 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:17.414576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822288c6-81e8-40a0-af1a-f14fc6b7e7db admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:17.431703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:17.433097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.433959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.434626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.436876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:17.441086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:17.472887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 705/1413] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:17.477320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:17.491259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5c35a-4bf0-41b0-99c1-94562de2edde admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:17.540330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3a88d-36d5-4009-a42f-71e15c4605ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:17.542214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3a88d-36d5-4009-a42f-71e15c4605ba admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.543282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3a88d-36d5-4009-a42f-71e15c4605ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.544066 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3a88d-36d5-4009-a42f-71e15c4605ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.546589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3a88d-36d5-4009-a42f-71e15c4605ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:17.555176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3a88d-36d5-4009-a42f-71e15c4605ba admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:17.596427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 709/1414] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:24:17.600687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:17.615878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048305-e371-4a2c-a31f-6da255557213 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:17.623052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba339fe-a436-431e-9bb5-b5c36fca9b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:17.623824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba339fe-a436-431e-9bb5-b5c36fca9b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.624174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba339fe-a436-431e-9bb5-b5c36fca9b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.624652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba339fe-a436-431e-9bb5-b5c36fca9b35 admin admin] PATH_INFO: `/v3/projects/75888ed189b44376ba2bd048999c9093/users/fe83c33274664b0b990c9e203f995605/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.660347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba339fe-a436-431e-9bb5-b5c36fca9b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75888ed189b44376ba2bd048999c9093, user_id=fe83c33274664b0b990c9e203f995605, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:17.663490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba339fe-a436-431e-9bb5-b5c36fca9b35 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:17.726824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 706/1415] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:17 2022] PUT /identity/v3/projects/75888ed189b44376ba2bd048999c9093/users/fe83c33274664b0b990c9e203f995605/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:17.737437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.737965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.738274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.783305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] MFA Rules not processed for user `fe83c33274664b0b990c9e203f995605`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:17.785599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:17.845688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d21ca-9768-420f-a6e5-ddc6495015f8 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.846119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d21ca-9768-420f-a6e5-ddc6495015f8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.846409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d21ca-9768-420f-a6e5-ddc6495015f8 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.847935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 707/1416] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 27 13:24:17.857398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.857593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.857876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.881018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 710/1417] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:17.889479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:17.891577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:17.894182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:17.895539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:17.950645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:17.978154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 708/1418] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:17.979347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34ca3b-5ad5-406b-82af-8335447d85fa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:17.984337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:17.985090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:17.985613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:17.986046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:17.991071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:17.992678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:17.997346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:17.998868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:18.006989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ea011-7e7e-42a8-bfb6-5ca2bbc8734e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:18.010306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 711/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:18.014451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa9686-9a0f-4082-881f-53e627b3ff9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:18.015219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa9686-9a0f-4082-881f-53e627b3ff9a service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:18.015530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa9686-9a0f-4082-881f-53e627b3ff9a service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:18.015802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa9686-9a0f-4082-881f-53e627b3ff9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:18.027347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa9686-9a0f-4082-881f-53e627b3ff9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:18.032676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa9686-9a0f-4082-881f-53e627b3ff9a service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:18.033031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:18.045691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 709/1420] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:18.106695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401321b-322b-4bb6-9553-5f6e98ff83af service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:18.111317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:18.112045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:18.112359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:18.112692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] PATH_INFO: `/v3/users/fe83c33274664b0b990c9e203f995605` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:18.113969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe83c33274664b0b990c9e203f995605)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:18.116032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:18.117731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:18.168604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe83c33274664b0b990c9e203f995605'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:18.181125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe83c33274664b0b990c9e203f995605'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:18.181893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe83c33274664b0b990c9e203f995605'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:18.193672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe83c33274664b0b990c9e203f995605'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:18.203407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe83c33274664b0b990c9e203f995605'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:18.209775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 712/1421] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:18 2022] DELETE /identity/v3/users/fe83c33274664b0b990c9e203f995605 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:18.213900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:18.214209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:18.215024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:18.234486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:18.235411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:18.312371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 710/1422] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:18.322090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:18.323629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:18.398136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69f4ee-1330-4896-a141-5a47af3500a6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:18.401906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d09c5a-b246-4d54-a229-02b4267bd836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:18.402526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d09c5a-b246-4d54-a229-02b4267bd836 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:18.402782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d09c5a-b246-4d54-a229-02b4267bd836 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:18.403052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d09c5a-b246-4d54-a229-02b4267bd836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:18.413137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d09c5a-b246-4d54-a229-02b4267bd836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:18.414562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d09c5a-b246-4d54-a229-02b4267bd836 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:18.424917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 713/1423] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:18.818293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:18.820447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:18.840136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf3739-e5c4-433a-bb6a-554c6bc56a98 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:18.848362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:18.849539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:18.850017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:18.850477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] PATH_INFO: `/v3/projects/75888ed189b44376ba2bd048999c9093` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:18.851888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75888ed189b44376ba2bd048999c9093)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:18.854756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:18.949397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75888ed189b44376ba2bd048999c9093', 'request_id': 'req-58b121dd-5e11-4b05-b1c7-c124be544e37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:18.965758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75888ed189b44376ba2bd048999c9093', 'request_id': 'req-58b121dd-5e11-4b05-b1c7-c124be544e37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:18.980447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75888ed189b44376ba2bd048999c9093', 'request_id': 'req-58b121dd-5e11-4b05-b1c7-c124be544e37'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:19.003653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] The token cache is being invalidate because project 75888ed189b44376ba2bd048999c9093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:19.004805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:19.005823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-58b121dd-5e11-4b05-b1c7-c124be544e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:19.009345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 711/1424] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:18 2022] DELETE /identity/v3/projects/75888ed189b44376ba2bd048999c9093 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:19.015253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3ba54-d37d-471a-acd7-2c62715bffb9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.015782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3ba54-d37d-471a-acd7-2c62715bffb9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.015782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3ba54-d37d-471a-acd7-2c62715bffb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.063621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-95b3ba54-d37d-471a-acd7-2c62715bffb9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:19.065386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-95b3ba54-d37d-471a-acd7-2c62715bffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:19.179910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 714/1425] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:19.182155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dc257-5968-4114-9bc9-c5ba63618d7c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.182356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dc257-5968-4114-9bc9-c5ba63618d7c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.182700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dc257-5968-4114-9bc9-c5ba63618d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.223016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e57dc257-5968-4114-9bc9-c5ba63618d7c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:19.228362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e57dc257-5968-4114-9bc9-c5ba63618d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:19.249749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 712/1426] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:19.251392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3ba54-d37d-471a-acd7-2c62715bffb9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:19.268090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3ba54-d37d-471a-acd7-2c62715bffb9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:19.272943 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:19.274160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.274160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.279687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.280418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:19.280728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:19.282347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:19.295003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 715/1427] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:19.299506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dc257-5968-4114-9bc9-c5ba63618d7c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:19.308955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dc257-5968-4114-9bc9-c5ba63618d7c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:19.314498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:19.315359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.315744 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.316116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.317897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:19.319853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:19.391410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 713/1428] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:19 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:19.392577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:19.407963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2e4e3-e908-4c6d-8cfd-bb8f4fda4465 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:19.418831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:19.503260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:19.504048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.504526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.505021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.505787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:19.507836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:19.544315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 716/1429] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:19 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:19.546217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:19.557435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b1e9f-2521-49a3-97da-af4e1f0ee25e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:19.563515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:19.564132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.564455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.564735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.565383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:19.566999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:19.581589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 714/1430] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:19.585844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:19.601199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e131a-a388-4fa3-9464-2386c1ef2aa8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:19.633590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff2cd5-c46a-46a0-9812-9d7cd9b5b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:19.639744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff2cd5-c46a-46a0-9812-9d7cd9b5b1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.640104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff2cd5-c46a-46a0-9812-9d7cd9b5b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.640409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff2cd5-c46a-46a0-9812-9d7cd9b5b1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.641109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff2cd5-c46a-46a0-9812-9d7cd9b5b1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:19.642921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff2cd5-c46a-46a0-9812-9d7cd9b5b1b1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:19.660142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 717/1431] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:19.664146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:19.677970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286b790-4a47-4bfe-ab7f-1a0c56cc20eb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:19.689429 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:19.697183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.697680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.698153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] PATH_INFO: `/v3/projects/5e96222073684dac86b68d5753cfa6d6/users/b8b6602042c642fa952318a095908b3f/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.727356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:19.733533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e96222073684dac86b68d5753cfa6d6, user_id=b8b6602042c642fa952318a095908b3f, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:19.737582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:19.753739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 715/1432] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:19 2022] PUT /identity/v3/projects/5e96222073684dac86b68d5753cfa6d6/users/b8b6602042c642fa952318a095908b3f/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:19.765502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:19.765809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:19.766112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:19.799713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] MFA Rules not processed for user `b8b6602042c642fa952318a095908b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:19.801051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:19.943171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 718/1433] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:19.958894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:19.960878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:20.094639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3835235-8379-46ec-9ac3-2a67df2f4c2b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:20.099518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f10dc9-ad8c-4869-9938-12c5e206159e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:20.100449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10dc9-ad8c-4869-9938-12c5e206159e service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:20.100945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10dc9-ad8c-4869-9938-12c5e206159e service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:20.101441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10dc9-ad8c-4869-9938-12c5e206159e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:20.127070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f10dc9-ad8c-4869-9938-12c5e206159e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:20.129922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f10dc9-ad8c-4869-9938-12c5e206159e service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:20.152859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 716/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:29.929709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:30.136388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:33.330000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:33.331486 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:33.530159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e17c1-2749-4cb7-ac8e-7ec2c9ce4c58 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:33.535084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:33.535618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:33.535892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:33.536157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] PATH_INFO: `/v3/users/94fc181fc107418fbc0ef34d91017f3a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:33.537442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fc181fc107418fbc0ef34d91017f3a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:33.539549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:33.603275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fc181fc107418fbc0ef34d91017f3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:33.617906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fc181fc107418fbc0ef34d91017f3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:33.618635 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fc181fc107418fbc0ef34d91017f3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:33.626421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fc181fc107418fbc0ef34d91017f3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:33.644912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fc181fc107418fbc0ef34d91017f3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:33.645152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 719/1435] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:33 2022] DELETE /identity/v3/users/94fc181fc107418fbc0ef34d91017f3a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:33.655969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:33.656500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:33.656646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:33.676149 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:33.677316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:33.738690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 717/1436] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:33.748316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:33.749930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:33.821322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25043ae2-d5c5-42f5-81aa-946c95a29695 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:33.825696 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:33.826535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:33.826877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:33.827212 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:33.837265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:33.838679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:33.848710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 720/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:34.201027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:34.202431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:34.214106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5229a0-02f9-407e-b701-75ff51243d4f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:34.218893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:34.219537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:34.219809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:34.220076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] PATH_INFO: `/v3/projects/4332bfb660d145f7bb81f02ed2c020ba` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:34.221288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4332bfb660d145f7bb81f02ed2c020ba)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:34.223168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:34.269834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4332bfb660d145f7bb81f02ed2c020ba', 'request_id': 'req-fd645929-1e88-4d0d-8a61-1d9cbf45c979'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.277048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4332bfb660d145f7bb81f02ed2c020ba', 'request_id': 'req-fd645929-1e88-4d0d-8a61-1d9cbf45c979'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.282467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4332bfb660d145f7bb81f02ed2c020ba', 'request_id': 'req-fd645929-1e88-4d0d-8a61-1d9cbf45c979'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.293319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] The token cache is being invalidate because project 4332bfb660d145f7bb81f02ed2c020ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:34.293597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.294213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.295666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 718/1438] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:34 2022] DELETE /identity/v3/projects/4332bfb660d145f7bb81f02ed2c020ba => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:34.299650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:34.309467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4c119-b284-48d5-83cc-306d7f623072 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:34.327971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:34.366199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:34.366826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:34.367084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:34.367352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] PATH_INFO: `/v3/users/fb32252da18943049ae90ca8fbab491b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:34.368487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb32252da18943049ae90ca8fbab491b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:34.370890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:34.414684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb32252da18943049ae90ca8fbab491b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.426741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb32252da18943049ae90ca8fbab491b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.427391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb32252da18943049ae90ca8fbab491b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.434711 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb32252da18943049ae90ca8fbab491b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.441522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-80f05a12-3c1b-46c3-89c2-2a3a33a91ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb32252da18943049ae90ca8fbab491b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.447212 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 721/1439] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:34 2022] DELETE /identity/v3/users/fb32252da18943049ae90ca8fbab491b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:34.779901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:34.781422 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:34.795492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd645929-1e88-4d0d-8a61-1d9cbf45c979 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:34.853862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:34.854491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:34.855333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:34.855601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] PATH_INFO: `/v3/projects/70c11efb1eb44853b12de09e7cdb2fc7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:34.856755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c11efb1eb44853b12de09e7cdb2fc7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:34.858662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:34.888584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:34.905922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c11efb1eb44853b12de09e7cdb2fc7', 'request_id': 'req-b6e1dcab-667d-4733-8b97-5cf615cca341'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.912686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c11efb1eb44853b12de09e7cdb2fc7', 'request_id': 'req-b6e1dcab-667d-4733-8b97-5cf615cca341'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.918587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c11efb1eb44853b12de09e7cdb2fc7', 'request_id': 'req-b6e1dcab-667d-4733-8b97-5cf615cca341'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.929373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] The token cache is being invalidate because project 70c11efb1eb44853b12de09e7cdb2fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:34.929652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.930272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b6e1dcab-667d-4733-8b97-5cf615cca341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:34.931752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 719/1440] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:34 2022] DELETE /identity/v3/projects/70c11efb1eb44853b12de09e7cdb2fc7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:34.936997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225f70f-4d2f-4af0-a8c4-351e341c1c5c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:34.937277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225f70f-4d2f-4af0-a8c4-351e341c1c5c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:34.937547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225f70f-4d2f-4af0-a8c4-351e341c1c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:34.956102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e225f70f-4d2f-4af0-a8c4-351e341c1c5c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:34.957296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e225f70f-4d2f-4af0-a8c4-351e341c1c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:35.016188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 722/1441] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:35.023843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7302a-5d83-4ad0-bf21-860454ed06f8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.024128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7302a-5d83-4ad0-bf21-860454ed06f8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.024425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7302a-5d83-4ad0-bf21-860454ed06f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.045754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a7c7302a-5d83-4ad0-bf21-860454ed06f8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:35.046959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a7c7302a-5d83-4ad0-bf21-860454ed06f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:35.055059 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 720/1442] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:35.059433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225f70f-4d2f-4af0-a8c4-351e341c1c5c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.069187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225f70f-4d2f-4af0-a8c4-351e341c1c5c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.073707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.074320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.074587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.074855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.075487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:35.075804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.077402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.093650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 723/1443] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:35.097957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7302a-5d83-4ad0-bf21-860454ed06f8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.108859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7302a-5d83-4ad0-bf21-860454ed06f8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.116106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.116770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.117065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.117429 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.117929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.119796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.158354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 721/1444] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:35 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:35.159406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.171378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b9329-b651-4ee9-a119-780357602062 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.208836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:35.255206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.256447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.256978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.257444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.258599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.262560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.302180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.303258 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 724/1445] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:35 2022] POST /identity/v3/users => generated 282 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:35.317818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f67605-cd7c-40e0-acbd-a953f92721d7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.327884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.328732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.329108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.329494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.330248 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.333494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.348103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 722/1446] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:35.350628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.361230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a20a1a-c82e-440c-9166-3efa0e4c665d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.367033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02340abe-feaa-4a48-84eb-aa88ef99cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.367687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02340abe-feaa-4a48-84eb-aa88ef99cd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.367911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02340abe-feaa-4a48-84eb-aa88ef99cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.368255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02340abe-feaa-4a48-84eb-aa88ef99cd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.368895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02340abe-feaa-4a48-84eb-aa88ef99cd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.370784 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02340abe-feaa-4a48-84eb-aa88ef99cd94 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.384518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 725/1447] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:35.386579 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.397670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2e811-fe54-4d33-ba01-7408ef02b74a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.408285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.408285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.408285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.408285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] PATH_INFO: `/v3/projects/f4f8d709987948ff8d655e20964eeaf2/users/9470346a6bb3464ebd1f97284f80cf21/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.428446 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f8d709987948ff8d655e20964eeaf2, user_id=9470346a6bb3464ebd1f97284f80cf21, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.432637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.460468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 723/1448] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:35 2022] PUT /identity/v3/projects/f4f8d709987948ff8d655e20964eeaf2/users/9470346a6bb3464ebd1f97284f80cf21/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:35.462218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a529b-e2cb-4ba2-b1de-feb1abed980c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.462572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a529b-e2cb-4ba2-b1de-feb1abed980c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.462825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a529b-e2cb-4ba2-b1de-feb1abed980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.505323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-785a529b-e2cb-4ba2-b1de-feb1abed980c None None] MFA Rules not processed for user `9470346a6bb3464ebd1f97284f80cf21`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:35.507211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-785a529b-e2cb-4ba2-b1de-feb1abed980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:35.614687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 726/1449] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:35.617588 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.727638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811e9e6-f2c2-4dc1-ac87-aaa2d8194ffe admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.733347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.734045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.734416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.734754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.735477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.737520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.776349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 724/1450] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:35 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:35.777545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a529b-e2cb-4ba2-b1de-feb1abed980c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.793782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a529b-e2cb-4ba2-b1de-feb1abed980c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:35.905124 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:35.906489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:35.906489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:35.906489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:35.907322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:35.909595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:35.944448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 727/1451] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:35 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:35.948664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:35.972130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080aaee-fd01-4fb9-8796-f31237f1204b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:36.004671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:36.004671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.004671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.004671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.004986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:36.004986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:36.015947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 725/1452] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:36.020265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:36.032275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec02a8-d49c-4876-bad0-03ede7c76963 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:36.038919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:36.039832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.040578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.040896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] PATH_INFO: `/v3/projects/d6920b867db44330a4ac8f9003f3fc3c/users/e08ab0cb56dc49aeae1e86d7cdee50bc/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.062865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6920b867db44330a4ac8f9003f3fc3c, user_id=e08ab0cb56dc49aeae1e86d7cdee50bc, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:36.065809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:36.080352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 728/1453] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:36 2022] PUT /identity/v3/projects/d6920b867db44330a4ac8f9003f3fc3c/users/e08ab0cb56dc49aeae1e86d7cdee50bc/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:36.084401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:36.094935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87497db2-1e43-42c2-b5a7-03663403346d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:36.115744 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:36.155760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a84a9-df3a-4e81-a3fb-5ecd7ffcb688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:36.156518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a84a9-df3a-4e81-a3fb-5ecd7ffcb688 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.156863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a84a9-df3a-4e81-a3fb-5ecd7ffcb688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.157203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a84a9-df3a-4e81-a3fb-5ecd7ffcb688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.157926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a84a9-df3a-4e81-a3fb-5ecd7ffcb688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:36.159619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a84a9-df3a-4e81-a3fb-5ecd7ffcb688 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:36.170019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 726/1454] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:36.175440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:36.190325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:36.199740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85070ff-d633-42e7-a62c-3231b7dfb0db admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:36.207920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:36.208710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.209123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.209392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] PATH_INFO: `/v3/projects/d6920b867db44330a4ac8f9003f3fc3c/users/e08ab0cb56dc49aeae1e86d7cdee50bc/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.211417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6920b867db44330a4ac8f9003f3fc3c, user_id=e08ab0cb56dc49aeae1e86d7cdee50bc, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:36.214203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:36.225321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 729/1455] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:36 2022] PUT /identity/v3/projects/d6920b867db44330a4ac8f9003f3fc3c/users/e08ab0cb56dc49aeae1e86d7cdee50bc/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:36.237420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77505a-3985-4218-a38a-f65c9a2234ca None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.237694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77505a-3985-4218-a38a-f65c9a2234ca None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.237966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77505a-3985-4218-a38a-f65c9a2234ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.267781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7c77505a-3985-4218-a38a-f65c9a2234ca None None] MFA Rules not processed for user `e08ab0cb56dc49aeae1e86d7cdee50bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:36.269029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7c77505a-3985-4218-a38a-f65c9a2234ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:36.344008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 727/1456] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:36.356187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:36.357889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:36.416990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478dda0-429c-493f-9dda-d4e0fba0d1ac admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:36.422932 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c430a2b-39d8-4261-9e95-04cee6ef023c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:36.423559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c430a2b-39d8-4261-9e95-04cee6ef023c service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.423902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c430a2b-39d8-4261-9e95-04cee6ef023c service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.424081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c430a2b-39d8-4261-9e95-04cee6ef023c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.435556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c430a2b-39d8-4261-9e95-04cee6ef023c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:36.437049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c430a2b-39d8-4261-9e95-04cee6ef023c service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:36.447941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 730/1457] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:36.868865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d55204-f5a6-4eaf-a128-aa280a0e8b6f None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.869327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d55204-f5a6-4eaf-a128-aa280a0e8b6f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.870275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d55204-f5a6-4eaf-a128-aa280a0e8b6f None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.872497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 728/1458] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:24:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:24:36.879318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:36.879819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:36.880516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:36.925325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:36.927112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:36.978911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d55204-f5a6-4eaf-a128-aa280a0e8b6f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:37.006530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 731/1459] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:24:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:37.064404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d55204-f5a6-4eaf-a128-aa280a0e8b6f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:37.069653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:37.070304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:37.072525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:37.072805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] PATH_INFO: `/v3/users/9470346a6bb3464ebd1f97284f80cf21` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:37.074590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9470346a6bb3464ebd1f97284f80cf21)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:37.081515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:37.146176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9470346a6bb3464ebd1f97284f80cf21'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.147565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:37.161275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9470346a6bb3464ebd1f97284f80cf21'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.162552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9470346a6bb3464ebd1f97284f80cf21'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.176632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9470346a6bb3464ebd1f97284f80cf21'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.182046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:37.185809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:37.188048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-317dac02-1351-4911-94bd-4a960eda6a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9470346a6bb3464ebd1f97284f80cf21'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.195696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 729/1460] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:36 2022] DELETE /identity/v3/users/9470346a6bb3464ebd1f97284f80cf21 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:37.200473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:37.200819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:37.201146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:37.202152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411a7b8-c527-4a9a-ad18-90c34bc8cfee None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:37.222372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:37.223496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:37.269676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:37.270469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:37.270767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:37.271059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:37.272885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:37.284596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:37.284911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 730/1461] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:37.286176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:37.294830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:37.296176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:37.296477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 732/1462] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:37.352797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94650d-e671-472e-bb53-3eb230c61fad None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:37.356275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:37.356874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:37.357145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:37.357410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:37.365957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:37.367361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:37.376933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 731/1463] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:37.640406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:37.641896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:37.657383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7905de6-e00e-42d8-9bf1-9da771023910 service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:37.668042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:37.669051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:37.669614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:37.670057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] PATH_INFO: `/v3/projects/f4f8d709987948ff8d655e20964eeaf2` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:37.671189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f8d709987948ff8d655e20964eeaf2)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:37.672246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: WARNING py.warnings [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 27 13:24:37.672246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: warnings.warn(msg) Oct 27 13:24:37.672246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: Oct 27 13:24:37.673501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:37.676157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:37.742419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f8d709987948ff8d655e20964eeaf2', 'request_id': 'req-bbcdc99b-7993-42b3-9359-642306976d70'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.753158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f8d709987948ff8d655e20964eeaf2', 'request_id': 'req-bbcdc99b-7993-42b3-9359-642306976d70'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.763652 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f8d709987948ff8d655e20964eeaf2', 'request_id': 'req-bbcdc99b-7993-42b3-9359-642306976d70'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.804472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] The token cache is being invalidate because project f4f8d709987948ff8d655e20964eeaf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:37.804472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.804472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.804472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 733/1464] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:37 2022] DELETE /identity/v3/projects/f4f8d709987948ff8d655e20964eeaf2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:37.809199 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:37.821751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c64d0f-6a04-4831-8d1d-f16ba3fe68b2 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:37.877964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:37.878517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:37.878792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:37.879066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] PATH_INFO: `/v3/users/e08ab0cb56dc49aeae1e86d7cdee50bc` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:37.880165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08ab0cb56dc49aeae1e86d7cdee50bc)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:37.882106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:37.892005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:37.930858 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08ab0cb56dc49aeae1e86d7cdee50bc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.942580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08ab0cb56dc49aeae1e86d7cdee50bc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.943249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08ab0cb56dc49aeae1e86d7cdee50bc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.949973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08ab0cb56dc49aeae1e86d7cdee50bc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.957177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e9560ab4-31ea-4006-b858-59dd3c7b11f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08ab0cb56dc49aeae1e86d7cdee50bc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:37.963025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 732/1465] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:37 2022] DELETE /identity/v3/users/e08ab0cb56dc49aeae1e86d7cdee50bc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:38.257040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:38.258286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:38.286488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdc99b-7993-42b3-9359-642306976d70 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:38.372809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:38.374131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:38.374570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:38.374970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] PATH_INFO: `/v3/projects/d6920b867db44330a4ac8f9003f3fc3c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:38.376397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6920b867db44330a4ac8f9003f3fc3c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:38.378363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:38.438936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6920b867db44330a4ac8f9003f3fc3c', 'request_id': 'req-ad4a08f3-b346-44d4-9d26-86a70727e8a0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:38.449530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6920b867db44330a4ac8f9003f3fc3c', 'request_id': 'req-ad4a08f3-b346-44d4-9d26-86a70727e8a0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:38.456399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6920b867db44330a4ac8f9003f3fc3c', 'request_id': 'req-ad4a08f3-b346-44d4-9d26-86a70727e8a0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:38.471496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] The token cache is being invalidate because project d6920b867db44330a4ac8f9003f3fc3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:38.471804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:38.472495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad4a08f3-b346-44d4-9d26-86a70727e8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:38.474408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 734/1466] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:38 2022] DELETE /identity/v3/projects/d6920b867db44330a4ac8f9003f3fc3c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:38.479187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4714799-1a32-4c0b-ac47-05193fcf4fe1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:38.479723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4714799-1a32-4c0b-ac47-05193fcf4fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:38.480891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4714799-1a32-4c0b-ac47-05193fcf4fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:38.510581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b4714799-1a32-4c0b-ac47-05193fcf4fe1 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:38.512577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b4714799-1a32-4c0b-ac47-05193fcf4fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:38.587179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 733/1467] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:38.596068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e2f9c-aab5-461a-a91d-e69dad2d98c9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:38.596846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e2f9c-aab5-461a-a91d-e69dad2d98c9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:38.597193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e2f9c-aab5-461a-a91d-e69dad2d98c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:38.630704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3e9e2f9c-aab5-461a-a91d-e69dad2d98c9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:38.632016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3e9e2f9c-aab5-461a-a91d-e69dad2d98c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:38.660742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 735/1468] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:38.666026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4714799-1a32-4c0b-ac47-05193fcf4fe1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:38.681476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4714799-1a32-4c0b-ac47-05193fcf4fe1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:38.688176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:38.689200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:38.689766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:38.690314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:38.691150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:38.691640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:38.693801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:38.715992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 734/1469] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:38.721033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e2f9c-aab5-461a-a91d-e69dad2d98c9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:38.733431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e2f9c-aab5-461a-a91d-e69dad2d98c9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:38.740113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:38.742469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:38.742959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:38.743632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:38.744643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:38.747394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:38.767661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:38.793904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 736/1470] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:38 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:38.796920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:38.825809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025cd1e-ecfb-4c5b-b0a3-9f93a53a4716 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:38.840513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:38.842497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:38.935417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba728a-e5d6-4cf8-8c19-4286d93d8c34 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:38.943201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:38.944540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:38.944540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:38.945700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] PATH_INFO: `/v3/users/b8b6602042c642fa952318a095908b3f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:38.947081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b6602042c642fa952318a095908b3f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:38.954191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:38.988782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:38.991270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:38.993887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:38.994373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:38.995827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.012904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.060774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b6602042c642fa952318a095908b3f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:39.063739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 735/1471] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:38 2022] POST /identity/v3/users => generated 294 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:39.070942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:39.083303 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b6602042c642fa952318a095908b3f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:39.085129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b6602042c642fa952318a095908b3f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:39.090693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:39.105761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5bb81-b098-4cad-9076-2b9d8909d92f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:39.115918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b6602042c642fa952318a095908b3f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:39.137300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-755cc4e2-4dfe-4636-b54b-c17ef858dd78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b6602042c642fa952318a095908b3f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:39.138499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:39.139418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.141163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.147076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.148144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.151835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.153019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 737/1472] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:38 2022] DELETE /identity/v3/users/b8b6602042c642fa952318a095908b3f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:39.155461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.155657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.156128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.184514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:39.186819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:39.208763 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 736/1473] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:39.213190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:39.215034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:39.245819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcbf37-2774-45e0-8865-bffba0ae8e7a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:39.297288 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 738/1474] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:39.311162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:39.311162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:39.328064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:39.357264 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:39.358156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.358465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.358739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.359602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.362339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.392376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c334ec-5498-46b2-b576-25c330e3d1f8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:39.394115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 737/1475] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:39.399326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:39.401044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c9b05-3202-4811-8341-bd2ef45181c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:39.402763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c9b05-3202-4811-8341-bd2ef45181c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.403532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c9b05-3202-4811-8341-bd2ef45181c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.404041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c9b05-3202-4811-8341-bd2ef45181c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.415243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c9b05-3202-4811-8341-bd2ef45181c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.417005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8e768-eda9-4935-a666-4954e5ec414d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:39.417936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c9b05-3202-4811-8341-bd2ef45181c3 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.422782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:39.423965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.424410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.425087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] PATH_INFO: `/v3/projects/6a328dab39314ccd92f20f141585764d/users/532f90950ee7466cac15c7b68af42d5b/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.433567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 739/1476] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:39.457080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a328dab39314ccd92f20f141585764d, user_id=532f90950ee7466cac15c7b68af42d5b, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.460790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.486319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 738/1477] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:39 2022] PUT /identity/v3/projects/6a328dab39314ccd92f20f141585764d/users/532f90950ee7466cac15c7b68af42d5b/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:39.498666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c09f2-49bc-4640-9556-0400119ce638 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.498952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c09f2-49bc-4640-9556-0400119ce638 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.499248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c09f2-49bc-4640-9556-0400119ce638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.535641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-437c09f2-49bc-4640-9556-0400119ce638 None None] MFA Rules not processed for user `532f90950ee7466cac15c7b68af42d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:39.537072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-437c09f2-49bc-4640-9556-0400119ce638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:39.653797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 740/1478] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:39.657813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:39.742037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2fb0-27bf-4c3a-ab71-8498ffe3b25b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:39.748648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:39.749372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.749719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.750054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.750811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.752827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.798682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 739/1479] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:39 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:39.803783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c09f2-49bc-4640-9556-0400119ce638 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:39.835790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:39.836747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c09f2-49bc-4640-9556-0400119ce638 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:39.837670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:39.924939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:39.934055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:39.934819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.935192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.935559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.936545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.938818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.943941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a0338-a3c0-4c98-93cc-04d68460c591 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:39.951859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:39.952651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:39.953179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:39.953460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] PATH_INFO: `/v3/projects/5e96222073684dac86b68d5753cfa6d6` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:39.954933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e96222073684dac86b68d5753cfa6d6)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:39.957395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:39.978972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 741/1480] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:39 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:39.980562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.015812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42d6d-61b3-4597-b872-1b440eceae71 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.034975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e96222073684dac86b68d5753cfa6d6', 'request_id': 'req-c81579fb-ce00-4efc-bad6-b18d593c8890'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:40.042277 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e96222073684dac86b68d5753cfa6d6', 'request_id': 'req-c81579fb-ce00-4efc-bad6-b18d593c8890'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:40.049100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e96222073684dac86b68d5753cfa6d6', 'request_id': 'req-c81579fb-ce00-4efc-bad6-b18d593c8890'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:40.057364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:40.063046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] The token cache is being invalidate because project 5e96222073684dac86b68d5753cfa6d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:40.063413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:40.064210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c81579fb-ce00-4efc-bad6-b18d593c8890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:40.065883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 740/1481] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:39 2022] DELETE /identity/v3/projects/5e96222073684dac86b68d5753cfa6d6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:40.071019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4838a2-e11f-4a13-b02f-74d3b9f7ad21 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.071291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4838a2-e11f-4a13-b02f-74d3b9f7ad21 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.071561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4838a2-e11f-4a13-b02f-74d3b9f7ad21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.089772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.090382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.090658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.090923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.091553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.093177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.093631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ed4838a2-e11f-4a13-b02f-74d3b9f7ad21 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:40.095002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ed4838a2-e11f-4a13-b02f-74d3b9f7ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:40.103883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 742/1482] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:40.104282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 741/1483] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:40.108392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.110669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:40.112341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180945-a5a5-401a-bb38-04642c87ed2e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.112630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180945-a5a5-401a-bb38-04642c87ed2e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.112890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180945-a5a5-401a-bb38-04642c87ed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.123702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aecd4-81fb-4fca-a400-59281732a6a6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.128740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.129335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.129610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.129894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.130517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.132095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.132526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4c180945-a5a5-401a-bb38-04642c87ed2e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:40.133736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4c180945-a5a5-401a-bb38-04642c87ed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:40.143231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 743/1484] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:24:40.144547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 742/1485] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:40.147698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.148585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c180945-a5a5-401a-bb38-04642c87ed2e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.157464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afcb3e-78a5-4525-8625-ab34f99dbad0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.157740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c180945-a5a5-401a-bb38-04642c87ed2e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.162049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86afaf2-a57f-4060-aaa8-21c4ead8ddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.162355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.162684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afaf2-a57f-4060-aaa8-21c4ead8ddb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.162936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.163020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afaf2-a57f-4060-aaa8-21c4ead8ddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.163210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.163377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afaf2-a57f-4060-aaa8-21c4ead8ddb9 admin admin] PATH_INFO: `/v3/projects/948309e50d9a4291b84346db78389596/users/f19b7dd4a8ca446a947d55dd67c13876/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.163461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.164097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:40.164421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.165996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.177675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 743/1486] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:24:40.179473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86afaf2-a57f-4060-aaa8-21c4ead8ddb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948309e50d9a4291b84346db78389596, user_id=f19b7dd4a8ca446a947d55dd67c13876, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.181993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.183075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86afaf2-a57f-4060-aaa8-21c4ead8ddb9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.193244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439ab5f-caae-4564-bac6-2677c98fc988 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.195612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 744/1487] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:40 2022] PUT /identity/v3/projects/948309e50d9a4291b84346db78389596/users/f19b7dd4a8ca446a947d55dd67c13876/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:40.207414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7581797-7991-4ab7-a7f6-e9a579ad6041 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.207810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7581797-7991-4ab7-a7f6-e9a579ad6041 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.208085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7581797-7991-4ab7-a7f6-e9a579ad6041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.271778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d7581797-7991-4ab7-a7f6-e9a579ad6041 None None] MFA Rules not processed for user `f19b7dd4a8ca446a947d55dd67c13876`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:40.276522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d7581797-7991-4ab7-a7f6-e9a579ad6041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:40.281362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.281989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.282265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.282530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.283174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.285887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.334538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 744/1488] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:40.339901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.400844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 745/1489] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:40.402125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7581797-7991-4ab7-a7f6-e9a579ad6041 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.424148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7581797-7991-4ab7-a7f6-e9a579ad6041 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.440312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:40.472581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ea180-a33c-4fc5-861a-c9d7810cf878 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.496058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.496923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.497241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.497510 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.498195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.500851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.528101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 745/1490] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:40.532613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.539715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.540497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.540843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.541185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.542579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.545595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.547984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d82e08-8a88-4bee-a30a-14bcd49d3bf7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.555900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.555900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.555900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.555900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.556633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.559045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.571716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 746/1491] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:40.575527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.603789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 746/1492] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:40.603879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c4800-8b72-4175-ae96-18c6d67b2de1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.609206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.645160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:40.713641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.714434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.714881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.715219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.717772 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.720031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.746397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 747/1493] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:40.747309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.761092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1eab-4961-435c-9b57-e878e8e3f189 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.783979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b6607-e018-408d-8bb7-be7da84d7f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.784657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b6607-e018-408d-8bb7-be7da84d7f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.784937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b6607-e018-408d-8bb7-be7da84d7f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.785212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b6607-e018-408d-8bb7-be7da84d7f82 admin admin] PATH_INFO: `/v3/projects/cd64533a5c0c4e35b0dd3d72e27fa366/users/16e59bcfddbd44c6832a230c6f196bbd/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.789492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9a418-8427-42a3-bafb-7d681ab61107 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.796170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.796825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.797104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.797373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.798007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.800069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.813517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b6607-e018-408d-8bb7-be7da84d7f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd64533a5c0c4e35b0dd3d72e27fa366, user_id=16e59bcfddbd44c6832a230c6f196bbd, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.816784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b6607-e018-408d-8bb7-be7da84d7f82 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.832362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 747/1494] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:40.848200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 748/1495] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:40 2022] PUT /identity/v3/projects/cd64533a5c0c4e35b0dd3d72e27fa366/users/16e59bcfddbd44c6832a230c6f196bbd/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:40.849390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.849934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf55a4-37ec-4474-85ff-f86283eae241 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.850219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf55a4-37ec-4474-85ff-f86283eae241 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.850494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf55a4-37ec-4474-85ff-f86283eae241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.861970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c600db-3dce-4ab3-8b0a-da492a7b6a12 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.884069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a6cf55a4-37ec-4474-85ff-f86283eae241 None None] MFA Rules not processed for user `16e59bcfddbd44c6832a230c6f196bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:40.885662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a6cf55a4-37ec-4474-85ff-f86283eae241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:40.949849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:40.950853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:40.951307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:40.951974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:40.952883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:40.954948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:40.972286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 748/1496] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:24:40.975791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.976587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 749/1497] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:40.978687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf55a4-37ec-4474-85ff-f86283eae241 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:40.990351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eefbe-7205-429e-8c00-33fb59db4773 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.992450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf55a4-37ec-4474-85ff-f86283eae241 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:40.995845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.000251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.001068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.001521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.002121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.003143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.006136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.010930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.011294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.011738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] PATH_INFO: `/v3/projects/b25a94563eb14d2f82cec64fe65d15a5/users/13bde533d30b4124ae4a849ca7bc1c7b/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.047179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:41.065015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 750/1498] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:40 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:41.066219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.075434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25a94563eb14d2f82cec64fe65d15a5, user_id=13bde533d30b4124ae4a849ca7bc1c7b, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.078575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.133508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 749/1499] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:40 2022] PUT /identity/v3/projects/b25a94563eb14d2f82cec64fe65d15a5/users/13bde533d30b4124ae4a849ca7bc1c7b/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:41.134437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.237734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df74c3a-96e8-4525-9bd5-c0decdce75e6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.259287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ff74-0aa9-411f-8ef2-89576a1e768a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.273767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.275078 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.276004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.281818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.282444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.284461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.314861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 750/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:41.317620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.342376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.342998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.343275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.343543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.344208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.346888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.347789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfe2a2-7486-4f40-8351-9964fbfb7fd1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.355814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295742b-a481-4e15-8c4b-47a611cf981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.357169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295742b-a481-4e15-8c4b-47a611cf981b admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.357453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295742b-a481-4e15-8c4b-47a611cf981b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.357726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295742b-a481-4e15-8c4b-47a611cf981b admin admin] PATH_INFO: `/v3/projects/b25a94563eb14d2f82cec64fe65d15a5/users/13bde533d30b4124ae4a849ca7bc1c7b/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.362766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295742b-a481-4e15-8c4b-47a611cf981b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25a94563eb14d2f82cec64fe65d15a5, user_id=13bde533d30b4124ae4a849ca7bc1c7b, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.367977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295742b-a481-4e15-8c4b-47a611cf981b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.378741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 751/1501] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:41 2022] POST /identity/v3/users => generated 288 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:41.378906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:41.379823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.390215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 751/1502] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:41 2022] PUT /identity/v3/projects/b25a94563eb14d2f82cec64fe65d15a5/users/13bde533d30b4124ae4a849ca7bc1c7b/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:41.392823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96101a2-59a9-4108-9fc4-e94bf5c96533 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.400759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8590b6-b069-497d-ade6-700daa5bbc63 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.401041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8590b6-b069-497d-ade6-700daa5bbc63 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.401905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8590b6-b069-497d-ade6-700daa5bbc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.430550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ce8590b6-b069-497d-ade6-700daa5bbc63 None None] MFA Rules not processed for user `13bde533d30b4124ae4a849ca7bc1c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:41.431741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ce8590b6-b069-497d-ade6-700daa5bbc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:41.453138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.453762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.454074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.454378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.455042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.456791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.467426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 752/1503] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:24:41.471335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.480844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086f66-1ad7-4eed-95fa-357b4ef269b2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.485421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.485977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.486285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.486597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.487235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.488842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.501070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 752/1504] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:41.504419 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 753/1505] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:41.509197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8590b6-b069-497d-ade6-700daa5bbc63 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.513389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.517219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:41.521365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8590b6-b069-497d-ade6-700daa5bbc63 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.525877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91451f0-f107-44da-9e1f-13427e613993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.527113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91451f0-f107-44da-9e1f-13427e613993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.527510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91451f0-f107-44da-9e1f-13427e613993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.527785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91451f0-f107-44da-9e1f-13427e613993 admin admin] PATH_INFO: `/v3/projects/f190b0e093c44722ba409483004765d7/users/ce0efd865fc44919b05e9ab554eeb897/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.538549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:41.544962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91451f0-f107-44da-9e1f-13427e613993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f190b0e093c44722ba409483004765d7, user_id=ce0efd865fc44919b05e9ab554eeb897, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.547599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91451f0-f107-44da-9e1f-13427e613993 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.568709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 753/1506] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:41 2022] PUT /identity/v3/projects/f190b0e093c44722ba409483004765d7/users/ce0efd865fc44919b05e9ab554eeb897/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:41.572059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ce5d-efc4-429a-9d48-f1da9ebbb946 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.572377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ce5d-efc4-429a-9d48-f1da9ebbb946 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.572752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ce5d-efc4-429a-9d48-f1da9ebbb946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.589487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47088552-83a9-4a75-a180-3f1cef55f37f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.606794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4530ce5d-efc4-429a-9d48-f1da9ebbb946 None None] MFA Rules not processed for user `ce0efd865fc44919b05e9ab554eeb897`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:41.608083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4530ce5d-efc4-429a-9d48-f1da9ebbb946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:41.646389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.647017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.647338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.647647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.679047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 754/1507] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:41.684439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530ce5d-efc4-429a-9d48-f1da9ebbb946 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.714894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.716397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.725671 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 754/1508] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:41.734848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:41.751034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530ce5d-efc4-429a-9d48-f1da9ebbb946 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.755987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.756500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.756757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.757022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.757639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.759501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.788008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 755/1509] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:41 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:41.791915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.802675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c40116-1f19-48c8-81fd-f20caa67d16c service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.895773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.896478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.896826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.897132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.897814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.900924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.936307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 755/1510] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:41 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:41.938406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:41.956176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282c6d3-5355-45fd-8841-4671da8cbb4d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:41.963083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:41.963745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:41.964026 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:41.964366 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:41.966907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:41.969289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:41.986092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 756/1511] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:41.987696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:42.021416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343993b-983f-44cd-beb5-ee14a64931fb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:42.032514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:42.034444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:42.034788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:42.035128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] PATH_INFO: `/v3/projects/4b67f7c014a6479d9b093ff5d4142fc8/users/90595bb936a64c6398e564626720b573/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:42.054986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:42.067683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b67f7c014a6479d9b093ff5d4142fc8, user_id=90595bb936a64c6398e564626720b573, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:42.072529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:42.107186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 756/1512] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:41 2022] PUT /identity/v3/projects/4b67f7c014a6479d9b093ff5d4142fc8/users/90595bb936a64c6398e564626720b573/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:42.111817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:42.138562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb806c65-9f55-4bf6-a6f1-023c9b8534ab admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:42.230298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a4e5a-3aa2-4be5-b706-faa0fb689148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:42.230926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4e5a-3aa2-4be5-b706-faa0fb689148 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:42.232358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4e5a-3aa2-4be5-b706-faa0fb689148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:42.232636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4e5a-3aa2-4be5-b706-faa0fb689148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:42.233299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a4e5a-3aa2-4be5-b706-faa0fb689148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:42.235471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a4e5a-3aa2-4be5-b706-faa0fb689148 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:42.260427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 757/1513] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:42.272180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:42.298554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695d12-d269-445d-89a0-a04c2d96c55c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:42.304387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:42.304848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:42.305169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:42.305545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] PATH_INFO: `/v3/projects/4b67f7c014a6479d9b093ff5d4142fc8/users/90595bb936a64c6398e564626720b573/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:42.314723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b67f7c014a6479d9b093ff5d4142fc8, user_id=90595bb936a64c6398e564626720b573, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:42.318238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:42.346737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 757/1514] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:42 2022] PUT /identity/v3/projects/4b67f7c014a6479d9b093ff5d4142fc8/users/90595bb936a64c6398e564626720b573/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:42.358718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded091c-0244-4250-ae07-d200274dadac None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:42.358984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded091c-0244-4250-ae07-d200274dadac None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:42.359252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded091c-0244-4250-ae07-d200274dadac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:42.417956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5ded091c-0244-4250-ae07-d200274dadac None None] MFA Rules not processed for user `90595bb936a64c6398e564626720b573`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:42.419194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5ded091c-0244-4250-ae07-d200274dadac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:42.579620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 758/1515] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:42.637788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:42.730336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e663d6-1846-4b8b-a0b7-768563d28119 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:42.748902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d0db4-6cf5-430f-82e5-d198d5e951a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:42.749550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d0db4-6cf5-430f-82e5-d198d5e951a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:42.751697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d0db4-6cf5-430f-82e5-d198d5e951a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:42.752020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d0db4-6cf5-430f-82e5-d198d5e951a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:42.837407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d0db4-6cf5-430f-82e5-d198d5e951a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:42.839885 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d0db4-6cf5-430f-82e5-d198d5e951a6 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:42.842117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded091c-0244-4250-ae07-d200274dadac None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:42.843552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5ded091c-0244-4250-ae07-d200274dadac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:42.861001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded091c-0244-4250-ae07-d200274dadac None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:42.869015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b8a10-96a5-4b83-b8a4-35cd8119e8ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:42.869509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b8a10-96a5-4b83-b8a4-35cd8119e8ff service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:42.869863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b8a10-96a5-4b83-b8a4-35cd8119e8ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:42.870262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b8a10-96a5-4b83-b8a4-35cd8119e8ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:42.872611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 758/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:42.874268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cd5b8a10-96a5-4b83-b8a4-35cd8119e8ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:42.942127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:42.973068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b8a10-96a5-4b83-b8a4-35cd8119e8ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:42.974680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b8a10-96a5-4b83-b8a4-35cd8119e8ff service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:42.985638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 759/1517] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:43.046948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc224b-7f95-45f3-924e-117b757010fe None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:43.047492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc224b-7f95-45f3-924e-117b757010fe None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:43.047840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc224b-7f95-45f3-924e-117b757010fe None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:43.050981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 759/1518] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:24:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:24:43.062889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:43.063044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:43.063416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:43.101897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:43.103122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:43.287136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 760/1519] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:43.302528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc224b-7f95-45f3-924e-117b757010fe None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:43.308760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-41fc224b-7f95-45f3-924e-117b757010fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:43.350760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc224b-7f95-45f3-924e-117b757010fe None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:43.383251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c55e1-e3bc-4ade-a090-9dc608cfc35d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:43.384262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c55e1-e3bc-4ade-a090-9dc608cfc35d service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:43.384888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c55e1-e3bc-4ade-a090-9dc608cfc35d service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:43.387212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c55e1-e3bc-4ade-a090-9dc608cfc35d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:43.415055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c55e1-e3bc-4ade-a090-9dc608cfc35d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:43.421270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c55e1-e3bc-4ade-a090-9dc608cfc35d service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:43.475762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 760/1520] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:43.501210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:43.502633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:43.641496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d942c-5ffe-435d-b541-5451a1fdc458 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:43.658557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:43.659981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:43.660289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:43.660575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] PATH_INFO: `/v3/users/3d012ea9fb264ba2b74df8dc803db381` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:43.661971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d012ea9fb264ba2b74df8dc803db381)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:43.663925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:43.736733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d012ea9fb264ba2b74df8dc803db381'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:43.765633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d012ea9fb264ba2b74df8dc803db381'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:43.767223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d012ea9fb264ba2b74df8dc803db381'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:43.777827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d012ea9fb264ba2b74df8dc803db381'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:43.780945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:43.796343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d012ea9fb264ba2b74df8dc803db381'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:43.808219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 761/1521] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:43 2022] DELETE /identity/v3/users/3d012ea9fb264ba2b74df8dc803db381 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:43.812940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:43.813339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:43.813534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:43.869569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:43.871328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:43.981626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 761/1522] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:43.995904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:44.001773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:44.077349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bb3b4-b496-4b24-adf2-25c756b14274 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:44.092473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:44.094098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:44.094454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:44.094809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:44.106740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:44.108621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:44.132181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 762/1523] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:44.471289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:44.472882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:44.489460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64586b80-6e15-4476-b5b0-e76394cdd259 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:44.494820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:44.495930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:44.496437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:44.496626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] PATH_INFO: `/v3/projects/3bae2cce8069489fa0d283e4011982b4` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:44.497808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bae2cce8069489fa0d283e4011982b4)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:44.506904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:44.524441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:44.559415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bae2cce8069489fa0d283e4011982b4', 'request_id': 'req-de7bef24-0e53-41a8-af00-6e7de6acd871'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.568617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bae2cce8069489fa0d283e4011982b4', 'request_id': 'req-de7bef24-0e53-41a8-af00-6e7de6acd871'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.575103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bae2cce8069489fa0d283e4011982b4', 'request_id': 'req-de7bef24-0e53-41a8-af00-6e7de6acd871'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.588152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] The token cache is being invalidate because project 3bae2cce8069489fa0d283e4011982b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:44.588501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.589169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.593612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 762/1524] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:44 2022] DELETE /identity/v3/projects/3bae2cce8069489fa0d283e4011982b4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:44.594937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:44.606990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e732e9b-6757-44bc-bd95-e109d69356f5 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:44.688276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:44.689174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:44.689629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:44.690317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] PATH_INFO: `/v3/users/ceef3d13766d4ee197eb954cd2be3e38` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:44.691852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceef3d13766d4ee197eb954cd2be3e38)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:44.695468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:44.729393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:44.731152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:44.766478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceef3d13766d4ee197eb954cd2be3e38'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.781057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceef3d13766d4ee197eb954cd2be3e38'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.786011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceef3d13766d4ee197eb954cd2be3e38'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.803101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceef3d13766d4ee197eb954cd2be3e38'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.813361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceef3d13766d4ee197eb954cd2be3e38'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:44.821469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 763/1525] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:44 2022] DELETE /identity/v3/users/ceef3d13766d4ee197eb954cd2be3e38 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:44.828709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bef24-0e53-41a8-af00-6e7de6acd871 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:44.903375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1e68e-3b9e-42e9-a1f6-ce5574e49581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:44.904010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1e68e-3b9e-42e9-a1f6-ce5574e49581 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:44.904322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1e68e-3b9e-42e9-a1f6-ce5574e49581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:44.904594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1e68e-3b9e-42e9-a1f6-ce5574e49581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:44.906538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eea1e68e-3b9e-42e9-a1f6-ce5574e49581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:44.970128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1e68e-3b9e-42e9-a1f6-ce5574e49581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:44.971566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1e68e-3b9e-42e9-a1f6-ce5574e49581 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:44.981169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 763/1526] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:45.133509 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.136554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:45.161812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:45.255133 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0636ef-6078-4a9b-8c6a-c1007c261ef3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:45.260923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:45.261679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.261822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.262099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] PATH_INFO: `/v3/projects/f435f558891348c3be4cb79a8dbc528d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.263850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f435f558891348c3be4cb79a8dbc528d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:45.266594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:45.332998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f435f558891348c3be4cb79a8dbc528d', 'request_id': 'req-0157aa2d-b06a-4a46-b2fd-37285edfc148'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:45.341819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f435f558891348c3be4cb79a8dbc528d', 'request_id': 'req-0157aa2d-b06a-4a46-b2fd-37285edfc148'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:45.360659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f435f558891348c3be4cb79a8dbc528d', 'request_id': 'req-0157aa2d-b06a-4a46-b2fd-37285edfc148'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:45.362550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] The token cache is being invalidate because project f435f558891348c3be4cb79a8dbc528d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:45.362927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:45.363698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0157aa2d-b06a-4a46-b2fd-37285edfc148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:45.369971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 764/1527] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:45 2022] DELETE /identity/v3/projects/f435f558891348c3be4cb79a8dbc528d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:45.371220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4347-b930-4b13-abd0-29aa56daba5f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.371502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4347-b930-4b13-abd0-29aa56daba5f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.371779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4347-b930-4b13-abd0-29aa56daba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.391779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-87ca4347-b930-4b13-abd0-29aa56daba5f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:45.392995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87ca4347-b930-4b13-abd0-29aa56daba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:45.402482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:45.459008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 764/1528] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:45.468623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d7cef-f206-44da-8a76-67aaf232ff97 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.468958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d7cef-f206-44da-8a76-67aaf232ff97 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.469233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d7cef-f206-44da-8a76-67aaf232ff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.489754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-468d7cef-f206-44da-8a76-67aaf232ff97 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:45.490958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-468d7cef-f206-44da-8a76-67aaf232ff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:45.499172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 765/1529] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:45.503174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4347-b930-4b13-abd0-29aa56daba5f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.513259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4347-b930-4b13-abd0-29aa56daba5f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:45.518255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:45.518942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.519296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.519566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.520207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:45.520532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:45.522101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:45.534391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 765/1530] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:45.538560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d7cef-f206-44da-8a76-67aaf232ff97 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.548412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d7cef-f206-44da-8a76-67aaf232ff97 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:45.553027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:45.553662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.553897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.554165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.554785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:45.556709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:45.583489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 766/1531] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:45 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:45.588098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.598867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b87af6-8e5d-493d-8f3f-5ec288cea7b7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:45.655724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:45.656365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.656641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.656909 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.657540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:45.659448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:45.681093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 766/1532] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:45 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:45.685428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.695622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006c168-60e3-4ea7-82f4-886e68691b7f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:45.700983 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:45.701584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.701860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.702126 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.702808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:45.704413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:45.715288 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 767/1533] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:45.719205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.729135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19488a4c-bd0a-4069-8ba8-ccd59cefe8ae admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:45.733606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1544e-8b1b-4f01-bfbe-e1147e15e61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:45.734196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1544e-8b1b-4f01-bfbe-e1147e15e61b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.734471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1544e-8b1b-4f01-bfbe-e1147e15e61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.734737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1544e-8b1b-4f01-bfbe-e1147e15e61b admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.735361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1544e-8b1b-4f01-bfbe-e1147e15e61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:45.736996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1544e-8b1b-4f01-bfbe-e1147e15e61b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:45.748300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 767/1534] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:45.752970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.764999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd72b7-291c-419e-99bf-345f0c1fb11f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:45.770096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:45.770877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.771158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.771521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] PATH_INFO: `/v3/projects/00e8ef706d134b8faf614a26a105ed6d/users/fcbe2dc0d9da4ad6a97d34c6f8cb7358/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.788577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e8ef706d134b8faf614a26a105ed6d, user_id=fcbe2dc0d9da4ad6a97d34c6f8cb7358, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:45.791198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:45.804995 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 768/1535] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:45 2022] PUT /identity/v3/projects/00e8ef706d134b8faf614a26a105ed6d/users/fcbe2dc0d9da4ad6a97d34c6f8cb7358/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:45.816713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed15a4c-18b9-4cec-9f0b-3b6f10e53523 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:45.817013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed15a4c-18b9-4cec-9f0b-3b6f10e53523 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:45.817300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed15a4c-18b9-4cec-9f0b-3b6f10e53523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:45.846362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eed15a4c-18b9-4cec-9f0b-3b6f10e53523 None None] MFA Rules not processed for user `fcbe2dc0d9da4ad6a97d34c6f8cb7358`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:45.847547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eed15a4c-18b9-4cec-9f0b-3b6f10e53523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:45.941889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 768/1536] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:45.946467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:45.989421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:46.012736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636240-76be-4eec-b082-78b5d87efb90 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:46.017515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:46.018119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.018396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.018663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.019285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:46.021195 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:46.049727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 769/1537] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:45 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:46.053991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed15a4c-18b9-4cec-9f0b-3b6f10e53523 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:46.064368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed15a4c-18b9-4cec-9f0b-3b6f10e53523 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:46.068818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:46.120740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:46.121347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.121630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.121901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.122549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:46.124476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:46.145427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 769/1538] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:46 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:46.149359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:46.159976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e49f38-6bdb-4c97-8b82-ff3deb80ca62 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:46.164589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:46.165183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.165462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.165731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.166361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:46.168068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:46.178359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 770/1539] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:46.182782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:46.193030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d454a-56be-434c-9ce4-6fd27ce1c0ed admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:46.197769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:46.198970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.199246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.199511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] PATH_INFO: `/v3/projects/e181b288e65b4c4296927e544ff54a22/users/f977d0f1ecfb45ad970d5dce4716f9d0/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.215373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e181b288e65b4c4296927e544ff54a22, user_id=f977d0f1ecfb45ad970d5dce4716f9d0, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:46.218068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:46.230002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 770/1540] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:46 2022] PUT /identity/v3/projects/e181b288e65b4c4296927e544ff54a22/users/f977d0f1ecfb45ad970d5dce4716f9d0/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:46.234041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:46.243761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f01c5-2b2f-4287-b3a4-59764bc6d1c1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:46.298265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65095633-2c44-4322-a899-795a0714ced3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:46.298867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65095633-2c44-4322-a899-795a0714ced3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.299145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65095633-2c44-4322-a899-795a0714ced3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.299416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65095633-2c44-4322-a899-795a0714ced3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.300046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65095633-2c44-4322-a899-795a0714ced3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:46.301666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65095633-2c44-4322-a899-795a0714ced3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:46.311526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 771/1541] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:46.315838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:46.326867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcf36b-c02e-4a6c-9131-6e7e3c7bfe45 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:46.331584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:46.332278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.332554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.332852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] PATH_INFO: `/v3/projects/e181b288e65b4c4296927e544ff54a22/users/f977d0f1ecfb45ad970d5dce4716f9d0/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.334763 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e181b288e65b4c4296927e544ff54a22, user_id=f977d0f1ecfb45ad970d5dce4716f9d0, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:46.337389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:46.347736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 771/1542] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:46 2022] PUT /identity/v3/projects/e181b288e65b4c4296927e544ff54a22/users/f977d0f1ecfb45ad970d5dce4716f9d0/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:46.359667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769146b-0ebe-4b30-9661-d047cc1fcaed None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.359948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769146b-0ebe-4b30-9661-d047cc1fcaed None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.360263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769146b-0ebe-4b30-9661-d047cc1fcaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.389397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8769146b-0ebe-4b30-9661-d047cc1fcaed None None] MFA Rules not processed for user `f977d0f1ecfb45ad970d5dce4716f9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:46.390588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8769146b-0ebe-4b30-9661-d047cc1fcaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:46.468817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 772/1543] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:46.479385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:46.481098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:46.548390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ee16e-4e21-40d5-ad33-59f46e2d4b70 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:46.552077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:46.552715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:46.552989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:46.553254 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:46.562327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:46.563735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:46.573705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 772/1544] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:47.011412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769146b-0ebe-4b30-9661-d047cc1fcaed None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:47.069822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:47.081174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68c3e9-1a28-4a4c-ba2c-1c33fbc8e0d0 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:47.081741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:47.086662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a48a3-4f81-4182-839a-6f144eae8bc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:47.087376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a48a3-4f81-4182-839a-6f144eae8bc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:47.087718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a48a3-4f81-4182-839a-6f144eae8bc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:47.087989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a48a3-4f81-4182-839a-6f144eae8bc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:47.089887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d93a48a3-4f81-4182-839a-6f144eae8bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:47.114318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769146b-0ebe-4b30-9661-d047cc1fcaed None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:47.120213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:47.120883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:47.121153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:47.121468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] PATH_INFO: `/v3/users/fcbe2dc0d9da4ad6a97d34c6f8cb7358` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:47.122595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbe2dc0d9da4ad6a97d34c6f8cb7358)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:47.124652 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:47.217057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a48a3-4f81-4182-839a-6f144eae8bc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:47.218764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a48a3-4f81-4182-839a-6f144eae8bc6 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:47.224157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:47.238860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbe2dc0d9da4ad6a97d34c6f8cb7358'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:47.241705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 773/1545] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:47.275311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbe2dc0d9da4ad6a97d34c6f8cb7358'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:47.279399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbe2dc0d9da4ad6a97d34c6f8cb7358'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:47.292439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbe2dc0d9da4ad6a97d34c6f8cb7358'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:47.310401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbe2dc0d9da4ad6a97d34c6f8cb7358'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:47.324379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 773/1546] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:47 2022] DELETE /identity/v3/users/fcbe2dc0d9da4ad6a97d34c6f8cb7358 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:47.330002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:47.330283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:47.330610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:47.368961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:47.371393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:47.451280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 774/1547] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:47.461179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:47.462622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:47.521697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8db775-2f33-4880-a3be-bcee71e148b5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:47.525515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:47.526810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:47.527080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:47.527343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:47.537378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:47.538877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:47.553264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 774/1548] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:48.174930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:48.176555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:48.196244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bd73b-a502-492f-b7cf-7ca3f4a5b652 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:48.209696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:48.210940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:48.211148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:48.212128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] PATH_INFO: `/v3/projects/00e8ef706d134b8faf614a26a105ed6d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:48.215933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e8ef706d134b8faf614a26a105ed6d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:48.217537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:48.267551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e8ef706d134b8faf614a26a105ed6d', 'request_id': 'req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.274184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e8ef706d134b8faf614a26a105ed6d', 'request_id': 'req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.281958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e8ef706d134b8faf614a26a105ed6d', 'request_id': 'req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.297464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] The token cache is being invalidate because project 00e8ef706d134b8faf614a26a105ed6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:48.297795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.308709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.311624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 775/1549] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:48 2022] DELETE /identity/v3/projects/00e8ef706d134b8faf614a26a105ed6d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:48.315330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:48.329986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5aa8-f63f-4a67-8957-53863bef7d30 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:48.407142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:48.413908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:48.414656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:48.414972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:48.415373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] PATH_INFO: `/v3/users/f977d0f1ecfb45ad970d5dce4716f9d0` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:48.416693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f977d0f1ecfb45ad970d5dce4716f9d0)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:48.419255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:48.501022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f977d0f1ecfb45ad970d5dce4716f9d0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.519113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f977d0f1ecfb45ad970d5dce4716f9d0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.520573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f977d0f1ecfb45ad970d5dce4716f9d0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.533609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f977d0f1ecfb45ad970d5dce4716f9d0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.550246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-932cfb94-f8a5-4aea-b552-3955745c45cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f977d0f1ecfb45ad970d5dce4716f9d0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:48.565888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 775/1550] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:48 2022] DELETE /identity/v3/users/f977d0f1ecfb45ad970d5dce4716f9d0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:49.023095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.024449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:49.040379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134d9c2-9c04-457f-b856-6c2e1cd5e4f8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.123065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.123720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.123999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.124295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] PATH_INFO: `/v3/projects/e181b288e65b4c4296927e544ff54a22` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.125432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e181b288e65b4c4296927e544ff54a22)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.127622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.161540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:49.192324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e181b288e65b4c4296927e544ff54a22', 'request_id': 'req-a147d94a-32b0-4eb4-99d1-e0bfff68f926'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:49.202829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e181b288e65b4c4296927e544ff54a22', 'request_id': 'req-a147d94a-32b0-4eb4-99d1-e0bfff68f926'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:49.209759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e181b288e65b4c4296927e544ff54a22', 'request_id': 'req-a147d94a-32b0-4eb4-99d1-e0bfff68f926'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:49.225848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] The token cache is being invalidate because project e181b288e65b4c4296927e544ff54a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:49.226196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:49.227685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a147d94a-32b0-4eb4-99d1-e0bfff68f926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:49.232320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 776/1551] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:49 2022] DELETE /identity/v3/projects/e181b288e65b4c4296927e544ff54a22 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:49.234803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9958863-c2e0-4160-8202-b557571134d0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.235235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9958863-c2e0-4160-8202-b557571134d0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.235573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9958863-c2e0-4160-8202-b557571134d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.256610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c9958863-c2e0-4160-8202-b557571134d0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:49.257827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c9958863-c2e0-4160-8202-b557571134d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:49.321912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 776/1552] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:49.330958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed5e3-900a-40ed-82b4-95108463178a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.331249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed5e3-900a-40ed-82b4-95108463178a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.331598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed5e3-900a-40ed-82b4-95108463178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.352289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-07fed5e3-900a-40ed-82b4-95108463178a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:49.353709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-07fed5e3-900a-40ed-82b4-95108463178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:49.361494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 777/1553] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:49.365730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9958863-c2e0-4160-8202-b557571134d0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.377015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9958863-c2e0-4160-8202-b557571134d0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.381298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.381917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.382235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.383103 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.392955 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:49.398086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.399766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.445332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 777/1554] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:49.457543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed5e3-900a-40ed-82b4-95108463178a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.475363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed5e3-900a-40ed-82b4-95108463178a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.487491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.488685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.488931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.489701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.490950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.492851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.536381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 778/1555] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:49 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:49.540729 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.554339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24759ee-fbba-48ce-9951-d6bfbe91e65d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.619419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.620953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.621322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.621598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.622390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.624405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.634655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:49.657978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 778/1556] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:49 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:49.659077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.664764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.666941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:49.669886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e721f85-d5a3-472e-8a17-ab48de7282c5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.674626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.675256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.675532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.675801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.676601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.678870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.690443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 779/1557] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:49.694652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.705735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252f5bd-593f-4df5-b513-aa112da95cb4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.710278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.710899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.711166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.711436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.712156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.713776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.725120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 780/1558] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:24:49.730150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:49.738618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b2496-51e0-45db-8687-a6294370edc0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.741442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c726a-8e16-4973-b4ab-0ce4bacf5c0f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:49.742524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.743144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.743434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.743690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.747260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:49.747551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787df08-deae-4b6e-a079-15f65469eca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:49.748469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787df08-deae-4b6e-a079-15f65469eca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.748754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787df08-deae-4b6e-a079-15f65469eca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.749134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787df08-deae-4b6e-a079-15f65469eca6 admin admin] PATH_INFO: `/v3/projects/398e497d82984f17b459a0f4d206d619/users/e898b0d6449c4a578253c6922efe0510/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.767077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787df08-deae-4b6e-a079-15f65469eca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=398e497d82984f17b459a0f4d206d619, user_id=e898b0d6449c4a578253c6922efe0510, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.770321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787df08-deae-4b6e-a079-15f65469eca6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.801258 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 781/1559] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:49 2022] PUT /identity/v3/projects/398e497d82984f17b459a0f4d206d619/users/e898b0d6449c4a578253c6922efe0510/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:49.808826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457a34-bd09-4b4c-b391-cb3205aafe72 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:49.809223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457a34-bd09-4b4c-b391-cb3205aafe72 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:49.809586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457a34-bd09-4b4c-b391-cb3205aafe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:49.828506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:49.830534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:49.840155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-26457a34-bd09-4b4c-b391-cb3205aafe72 None None] MFA Rules not processed for user `e898b0d6449c4a578253c6922efe0510`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:49.841506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-26457a34-bd09-4b4c-b391-cb3205aafe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:49.843441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 779/1560] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:50.089805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 782/1561] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:50.125493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:50.127117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:50.140611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050902b-0062-4552-b379-3b7e1a16fa55 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:50.218242 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760c8f0-80d3-4148-91fe-f2872d9b86be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:50.218870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760c8f0-80d3-4148-91fe-f2872d9b86be service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:50.219146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760c8f0-80d3-4148-91fe-f2872d9b86be service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:50.219417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760c8f0-80d3-4148-91fe-f2872d9b86be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:50.241562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760c8f0-80d3-4148-91fe-f2872d9b86be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:50.243033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760c8f0-80d3-4148-91fe-f2872d9b86be service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:50.253641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 780/1562] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:50.280561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457a34-bd09-4b4c-b391-cb3205aafe72 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:50.359132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:50.406112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457a34-bd09-4b4c-b391-cb3205aafe72 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:50.413180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:50.414110 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:50.414666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:50.415042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] PATH_INFO: `/v3/users/e898b0d6449c4a578253c6922efe0510` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:50.416934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e898b0d6449c4a578253c6922efe0510)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:50.419939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:50.517913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e898b0d6449c4a578253c6922efe0510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:50.549495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e898b0d6449c4a578253c6922efe0510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:50.550323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e898b0d6449c4a578253c6922efe0510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:50.568875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e898b0d6449c4a578253c6922efe0510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:50.597125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e898b0d6449c4a578253c6922efe0510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:50.606622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 783/1563] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:50 2022] DELETE /identity/v3/users/e898b0d6449c4a578253c6922efe0510 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:50.611177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:50.611752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:50.611752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:50.650142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:50.654684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:50.727482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:50.753841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 781/1564] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:50.769798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:50.771762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:50.908311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f56d4e-665c-4167-acd6-57ddb299d18b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:50.912990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:50.913913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:50.914314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:50.914851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:50.930863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:50.933772 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:50.958735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 784/1565] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:51.449406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:51.451917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:51.474879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b6b58-f565-4319-8e42-438c67733ea8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:51.481025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:51.481895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.482167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.485562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] PATH_INFO: `/v3/projects/398e497d82984f17b459a0f4d206d619` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.486616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:51.487053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=398e497d82984f17b459a0f4d206d619)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:51.488039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:51.497900 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:51.507750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85161831-a741-4a6b-8ffe-e88e2c7cca00 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:51.522577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:51.523225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.523500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.523769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] PATH_INFO: `/v3/users/2e7ebd94fa1043fc9df1d6b58f69dc38` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.525358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7ebd94fa1043fc9df1d6b58f69dc38)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:51.530271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:51.572839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398e497d82984f17b459a0f4d206d619', 'request_id': 'req-406806f0-6ff5-453d-b07c-2129d42ef70b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.596948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398e497d82984f17b459a0f4d206d619', 'request_id': 'req-406806f0-6ff5-453d-b07c-2129d42ef70b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.603558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:51.608432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398e497d82984f17b459a0f4d206d619', 'request_id': 'req-406806f0-6ff5-453d-b07c-2129d42ef70b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.608954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7ebd94fa1043fc9df1d6b58f69dc38'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.631485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] The token cache is being invalidate because project 398e497d82984f17b459a0f4d206d619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:51.631953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.632799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-406806f0-6ff5-453d-b07c-2129d42ef70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.637388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 782/1566] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:51 2022] DELETE /identity/v3/projects/398e497d82984f17b459a0f4d206d619 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:51.642579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0ebb4-e58e-4af6-9da2-39fc18194962 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.642982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0ebb4-e58e-4af6-9da2-39fc18194962 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.643358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0ebb4-e58e-4af6-9da2-39fc18194962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.652033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7ebd94fa1043fc9df1d6b58f69dc38'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.652906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7ebd94fa1043fc9df1d6b58f69dc38'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.670167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7ebd94fa1043fc9df1d6b58f69dc38'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.670317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-15d0ebb4-e58e-4af6-9da2-39fc18194962 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:51.671707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-15d0ebb4-e58e-4af6-9da2-39fc18194962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:51.679647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a171c1bc-ef86-4349-ae3e-9486e436d692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7ebd94fa1043fc9df1d6b58f69dc38'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:51.686849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 785/1567] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:51 2022] DELETE /identity/v3/users/2e7ebd94fa1043fc9df1d6b58f69dc38 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:51.692714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.693737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.694187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.718055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:51.719531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:51.743426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 783/1568] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:51.752474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcbdf3-3f37-44e1-a0c4-731b30df72da None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.752761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcbdf3-3f37-44e1-a0c4-731b30df72da None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.753032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcbdf3-3f37-44e1-a0c4-731b30df72da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.773625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9dbcbdf3-3f37-44e1-a0c4-731b30df72da None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:51.775064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9dbcbdf3-3f37-44e1-a0c4-731b30df72da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:51.785510 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 786/1569] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:51.796021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:51.797475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:51.842511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 784/1570] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:51.846653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcbdf3-3f37-44e1-a0c4-731b30df72da None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:51.855236 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcbdf3-3f37-44e1-a0c4-731b30df72da None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:51.858651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0d5e6-c135-4159-9e88-f58d9312ca7c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:51.859693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:51.860318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.860569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.860832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.861461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:51.861755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:51.862252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:51.862858 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.863136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.863345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:51.863459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.872781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:51.874211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:51.874417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 785/1571] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:24:51.878667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:51.883834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:51.884166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 787/1572] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:51.890697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb5288-c5a2-4e68-8b6f-52535d04800d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:51.896874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:51.897480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:51.897757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:51.898025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:51.898653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:51.900671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:51.931806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 786/1573] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:51 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:51.933378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:51.943768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e651-c80a-47a3-bba2-f83c2aafc1bf service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:52.026557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:52.027424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.027710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.027985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.028657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:52.030722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:52.047098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:52.069599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 788/1574] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:51 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:52.076269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.105591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb68b0-5fdd-4ca5-829f-82c8ee003774 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:52.111914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:52.112643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.112966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.113230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.113917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:52.116684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:52.132007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 787/1575] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:52.134127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.151714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab691f-36e9-4dbd-8d9c-da01a6792c23 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:52.158621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18f3bf-5b2d-48fc-a8cd-fa4c45a30d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:52.159333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18f3bf-5b2d-48fc-a8cd-fa4c45a30d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.159787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18f3bf-5b2d-48fc-a8cd-fa4c45a30d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.160069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18f3bf-5b2d-48fc-a8cd-fa4c45a30d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.160910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18f3bf-5b2d-48fc-a8cd-fa4c45a30d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:52.162727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18f3bf-5b2d-48fc-a8cd-fa4c45a30d89 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:52.175296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 789/1576] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:52.179634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.190961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63941e5c-1c7e-4954-816a-6a19af947e21 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:52.195580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:52.196252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.196526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.196790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] PATH_INFO: `/v3/projects/aea0cf9918594b1bbaee3a89da127eff/users/2cbfebb0012d45478daf865dbb3b8209/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.212630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea0cf9918594b1bbaee3a89da127eff, user_id=2cbfebb0012d45478daf865dbb3b8209, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:52.215473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:52.228685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 788/1577] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:52 2022] PUT /identity/v3/projects/aea0cf9918594b1bbaee3a89da127eff/users/2cbfebb0012d45478daf865dbb3b8209/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:52.239047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.239257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.239533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.269631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] MFA Rules not processed for user `2cbfebb0012d45478daf865dbb3b8209`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:52.271353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:52.339598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.341872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:52.360742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 790/1578] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:52.374723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.376566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:52.440561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d329ba-4d2c-4722-aae9-11f16d70fe4d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:52.446796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:52.447516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.447801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.448100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] PATH_INFO: `/v3/users/16e59bcfddbd44c6832a230c6f196bbd` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.449611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e59bcfddbd44c6832a230c6f196bbd)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:52.451503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:52.504706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e59bcfddbd44c6832a230c6f196bbd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.505499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb734f7e-42fa-40da-a6df-9f9ad129fdda None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:52.510681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d4ce3e02e5945ffbbde389398b146c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:52.511297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.511619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.511891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.521652 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e59bcfddbd44c6832a230c6f196bbd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.521897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:52.522832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e59bcfddbd44c6832a230c6f196bbd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.523532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:52.532837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e59bcfddbd44c6832a230c6f196bbd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.534598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 791/1579] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:52.536695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.539311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:52.571729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386a93f-7381-4c1d-b97a-eccbd862f241 service glance] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:52.572532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-008845ae-beea-40e0-95f5-cd2ea50a7fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e59bcfddbd44c6832a230c6f196bbd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.577266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:52.578176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.578793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.579079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] PATH_INFO: `/v3/projects/ba958741e9c24c729a5cff811019511f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.580219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba958741e9c24c729a5cff811019511f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:52.582162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:52.615070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 789/1580] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:52 2022] DELETE /identity/v3/users/16e59bcfddbd44c6832a230c6f196bbd => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:52.616948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:52.617233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:52.621534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:52.653827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:52.655179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:52.686880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:52.742994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba958741e9c24c729a5cff811019511f', 'request_id': 'req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.746950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=1,keystone:SELECT=95,keystone:DELETE=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:52.764634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba958741e9c24c729a5cff811019511f', 'request_id': 'req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.774392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba958741e9c24c729a5cff811019511f', 'request_id': 'req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.784242 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 790/1581] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:52.796139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.798012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:52.814136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] The token cache is being invalidate because project ba958741e9c24c729a5cff811019511f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:52.814431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.816602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:52.819418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 792/1582] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:52 2022] DELETE /identity/v3/projects/ba958741e9c24c729a5cff811019511f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:52.823831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:52.831442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:52.894439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5007ed-3503-4af1-96f0-33c60d75d910 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:53.008145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:53.008908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:53.009197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:53.009597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:53.011205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5b0dc-f067-442c-bdc8-76a248e5a7eb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:53.012743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:53.016307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:53.017836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:53.018119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:53.018401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] PATH_INFO: `/v3/users/bf5909a23fd74aa0982319c2715289d5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:53.020744 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5909a23fd74aa0982319c2715289d5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:53.024949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:53.048330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:53.049959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:53.062784 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 791/1583] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:24:53.063756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:53.065235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:53.079516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89107ab8-0ca0-4789-b2d9-0adcd006dec0 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:53.086588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:53.087406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:53.087687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:53.088842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] PATH_INFO: `/v3/users/532f90950ee7466cac15c7b68af42d5b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:53.090361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] RBAC: Authorizing `identity:delete_user(user_id=532f90950ee7466cac15c7b68af42d5b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:53.092990 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:53.095246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5909a23fd74aa0982319c2715289d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.124563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5909a23fd74aa0982319c2715289d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.126993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5909a23fd74aa0982319c2715289d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.161240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532f90950ee7466cac15c7b68af42d5b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.163873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5909a23fd74aa0982319c2715289d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.183804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80fcfca4-2e44-4c08-8563-2f0ebc1ab22d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5909a23fd74aa0982319c2715289d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.188572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532f90950ee7466cac15c7b68af42d5b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.194923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 793/1584] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:52 2022] DELETE /identity/v3/users/bf5909a23fd74aa0982319c2715289d5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:53.198607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532f90950ee7466cac15c7b68af42d5b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.205268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532f90950ee7466cac15c7b68af42d5b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.210866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:53.222203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532f90950ee7466cac15c7b68af42d5b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:53.235947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 792/1585] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:53 2022] DELETE /identity/v3/users/532f90950ee7466cac15c7b68af42d5b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:53.241382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:53.242165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:53.242544 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:53.275080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:53.277719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:53.425982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:53.465846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 794/1586] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:53.497234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:53.500843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:53.601053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e33ac-e83e-4857-9bfa-3da0d6d2930c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:53.609882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:53.610768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:53.611246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:53.611926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:53.625369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:53.627132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:53.640010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 793/1587] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:54.082627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:54.087851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:54.103510 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94b750-d032-43d3-b5f7-d9859772df48 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:54.109261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:54.109912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:54.110217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:54.110450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] PATH_INFO: `/v3/projects/6a328dab39314ccd92f20f141585764d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:54.111606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a328dab39314ccd92f20f141585764d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:54.113604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:54.183904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:54.185739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:54.187121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a328dab39314ccd92f20f141585764d', 'request_id': 'req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.201990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a328dab39314ccd92f20f141585764d', 'request_id': 'req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.209154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0de1f3-d9c3-4aeb-8760-8ce80a4867a8 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:54.213761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a328dab39314ccd92f20f141585764d', 'request_id': 'req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.238283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] The token cache is being invalidate because project 6a328dab39314ccd92f20f141585764d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:54.238697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.239894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.243252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 795/1588] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:54 2022] DELETE /identity/v3/projects/6a328dab39314ccd92f20f141585764d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:54.248187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:54.250848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:54.295675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:54.296437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:54.296782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:54.297137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] PATH_INFO: `/v3/projects/cd64533a5c0c4e35b0dd3d72e27fa366` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:54.298534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd64533a5c0c4e35b0dd3d72e27fa366)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:54.301999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:54.346066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba5388-3a62-43ae-8e6a-a49b19c90d9b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:54.357282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:54.358241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:54.358566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:54.358835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] PATH_INFO: `/v3/projects/dc8b28be83964891930feb91a3f62b6e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:54.360175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8b28be83964891930feb91a3f62b6e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:54.362502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:54.402713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd64533a5c0c4e35b0dd3d72e27fa366', 'request_id': 'req-5bb27d1c-44ba-45d7-802b-e71a289a753c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.415215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd64533a5c0c4e35b0dd3d72e27fa366', 'request_id': 'req-5bb27d1c-44ba-45d7-802b-e71a289a753c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.423041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd64533a5c0c4e35b0dd3d72e27fa366', 'request_id': 'req-5bb27d1c-44ba-45d7-802b-e71a289a753c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.436119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] The token cache is being invalidate because project cd64533a5c0c4e35b0dd3d72e27fa366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:54.436450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.437139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.440592 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 794/1589] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:54 2022] DELETE /identity/v3/projects/cd64533a5c0c4e35b0dd3d72e27fa366 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:54.440756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:54.442571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:54.465533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8b28be83964891930feb91a3f62b6e', 'request_id': 'req-82013c1a-6408-4ca2-bab7-752a788db8c5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.484465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8b28be83964891930feb91a3f62b6e', 'request_id': 'req-82013c1a-6408-4ca2-bab7-752a788db8c5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.503255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8b28be83964891930feb91a3f62b6e', 'request_id': 'req-82013c1a-6408-4ca2-bab7-752a788db8c5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.519570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:54.544612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] The token cache is being invalidate because project dc8b28be83964891930feb91a3f62b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:54.545161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.549613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.552868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 796/1590] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:54 2022] DELETE /identity/v3/projects/dc8b28be83964891930feb91a3f62b6e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:54.553963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:54.560310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:54.570892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb27d1c-44ba-45d7-802b-e71a289a753c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:54.578564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:54.680925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:54.682476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:54.682953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:54.683313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] PATH_INFO: `/v3/users/f19b7dd4a8ca446a947d55dd67c13876` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:54.684785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19b7dd4a8ca446a947d55dd67c13876)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:54.686986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:54.691061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82013c1a-6408-4ca2-bab7-752a788db8c5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:54.699214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:54.699952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:54.700382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:54.701293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] PATH_INFO: `/v3/users/ce0efd865fc44919b05e9ab554eeb897` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:54.702566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce0efd865fc44919b05e9ab554eeb897)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:54.705758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:54.764626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19b7dd4a8ca446a947d55dd67c13876'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.782147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19b7dd4a8ca446a947d55dd67c13876'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.782851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19b7dd4a8ca446a947d55dd67c13876'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.795315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19b7dd4a8ca446a947d55dd67c13876'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.805781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0463c1d0-21e9-47ce-bf20-ffe4e689824e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19b7dd4a8ca446a947d55dd67c13876'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.805970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce0efd865fc44919b05e9ab554eeb897'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.816593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 795/1591] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:54 2022] DELETE /identity/v3/users/f19b7dd4a8ca446a947d55dd67c13876 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:54.818570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02706-1539-484a-b16a-65f8b4d372a5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:54.819156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02706-1539-484a-b16a-65f8b4d372a5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:54.819583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02706-1539-484a-b16a-65f8b4d372a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:54.821330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce0efd865fc44919b05e9ab554eeb897'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.822066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce0efd865fc44919b05e9ab554eeb897'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.837650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce0efd865fc44919b05e9ab554eeb897'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.850525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6f0a1b20-7874-4534-bb2c-2b1c2c319225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce0efd865fc44919b05e9ab554eeb897'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:54.859505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-12a02706-1539-484a-b16a-65f8b4d372a5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:54.861067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-12a02706-1539-484a-b16a-65f8b4d372a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:54.867048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 797/1592] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:54 2022] DELETE /identity/v3/users/ce0efd865fc44919b05e9ab554eeb897 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:54.925626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:54.949837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 796/1593] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:54.953721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e90a-8d41-4039-b36b-82684e5a4c37 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:54.954507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e90a-8d41-4039-b36b-82684e5a4c37 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:54.954507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e90a-8d41-4039-b36b-82684e5a4c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.007689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f630e90a-8d41-4039-b36b-82684e5a4c37 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:55.009321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f630e90a-8d41-4039-b36b-82684e5a4c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:55.093145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 798/1594] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:55.097423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02706-1539-484a-b16a-65f8b4d372a5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.110569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02706-1539-484a-b16a-65f8b4d372a5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.116481 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.117587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.117904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.118172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.118901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:55.119391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.121076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.143870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 797/1595] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:24:55.145120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630e90a-8d41-4039-b36b-82684e5a4c37 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.156159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630e90a-8d41-4039-b36b-82684e5a4c37 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.166724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.167649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.167988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.168349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.169073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.171205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.222510 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 799/1596] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:55 2022] POST /identity/v3/projects => generated 355 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:55.226936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.238591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca573b7-95b1-4ba0-8bf6-13bfedf9283f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.358985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.359602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.359883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.360134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.360809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.362970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.370098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.376751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:55.391736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 798/1597] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:55 2022] POST /identity/v3/users => generated 278 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:55.396012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.407921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81afb1f-bf8b-4daf-a398-ba8601880d4b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.413725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.414686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.414686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.415091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.416094 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.417763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.433949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 799/1598] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:55.434896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.446738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb06a6-bac8-444a-8621-b39692627006 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.447104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ec8ba-ad3a-465d-b1bb-931f751bf5b7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.452295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.452767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.452893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.453201 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.453478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.453580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.453895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.454228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.454347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] PATH_INFO: `/v3/projects/948309e50d9a4291b84346db78389596` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.455421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=948309e50d9a4291b84346db78389596)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.455876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.457498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.460590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:55.471032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 800/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:55.475440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.487338 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a7d4c-bb7a-445f-8c9f-a268676d1f34 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.493799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.494502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.494887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.495161 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] PATH_INFO: `/v3/projects/99b150fa597a437a97ed50393637d3e2/users/84215a77e65248099434dc4d9884ba7a/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.525125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b150fa597a437a97ed50393637d3e2, user_id=84215a77e65248099434dc4d9884ba7a, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.529335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.533474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948309e50d9a4291b84346db78389596', 'request_id': 'req-79a75e82-16c1-42ee-a755-8673669ad851'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.552318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 801/1600] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:55 2022] PUT /identity/v3/projects/99b150fa597a437a97ed50393637d3e2/users/84215a77e65248099434dc4d9884ba7a/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:55.553632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948309e50d9a4291b84346db78389596', 'request_id': 'req-79a75e82-16c1-42ee-a755-8673669ad851'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.553856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.555896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:55.576200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948309e50d9a4291b84346db78389596', 'request_id': 'req-79a75e82-16c1-42ee-a755-8673669ad851'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.599048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] The token cache is being invalidate because project 948309e50d9a4291b84346db78389596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:55.599238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.599985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-79a75e82-16c1-42ee-a755-8673669ad851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.604578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 800/1601] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:55 2022] DELETE /identity/v3/projects/948309e50d9a4291b84346db78389596 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:55.604578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.604578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.604578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.650328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4d4f8-a5f2-4d4c-a4f6-5e9991fc32c6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.656618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.657735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.658241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.658853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] PATH_INFO: `/v3/projects/f190b0e093c44722ba409483004765d7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.660143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f190b0e093c44722ba409483004765d7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.662283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] MFA Rules not processed for user `84215a77e65248099434dc4d9884ba7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:55.662428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.663615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:55.703285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:55.730860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f190b0e093c44722ba409483004765d7', 'request_id': 'req-3f69e37e-8c54-4db7-8406-e7547344e9ad'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.745700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f190b0e093c44722ba409483004765d7', 'request_id': 'req-3f69e37e-8c54-4db7-8406-e7547344e9ad'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.761023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f190b0e093c44722ba409483004765d7', 'request_id': 'req-3f69e37e-8c54-4db7-8406-e7547344e9ad'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.791790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] The token cache is being invalidate because project f190b0e093c44722ba409483004765d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:55.792064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.792863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:55.794271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 802/1602] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:55 2022] DELETE /identity/v3/projects/f190b0e093c44722ba409483004765d7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:55.795031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.796630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:55.799487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 801/1603] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:55.800581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:55.804872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:55.824936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f419a8-1fee-4438-9085-59dbea77d233 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.907956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.920295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.920619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.920896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] PATH_INFO: `/v3/users/90595bb936a64c6398e564626720b573` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.922665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=90595bb936a64c6398e564626720b573)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.928564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.931575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69e37e-8c54-4db7-8406-e7547344e9ad admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:55.947400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:55.953611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:55.954092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:55.954382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] PATH_INFO: `/v3/users/13bde533d30b4124ae4a849ca7bc1c7b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:55.955725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=13bde533d30b4124ae4a849ca7bc1c7b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:55.957949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:55.966525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:56.015942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90595bb936a64c6398e564626720b573'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.056220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90595bb936a64c6398e564626720b573'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.058904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90595bb936a64c6398e564626720b573'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.062139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13bde533d30b4124ae4a849ca7bc1c7b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.078268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90595bb936a64c6398e564626720b573'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.089903 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13bde533d30b4124ae4a849ca7bc1c7b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.091050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13bde533d30b4124ae4a849ca7bc1c7b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.097395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90595bb936a64c6398e564626720b573'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.110670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 802/1604] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:55 2022] DELETE /identity/v3/users/90595bb936a64c6398e564626720b573 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:56.123826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:56.131986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:56.132910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13bde533d30b4124ae4a849ca7bc1c7b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.159969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13bde533d30b4124ae4a849ca7bc1c7b'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:56.180946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 803/1605] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:24:55 2022] DELETE /identity/v3/users/13bde533d30b4124ae4a849ca7bc1c7b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:56.263537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9fca2-f585-4e23-94a1-1a467e6eb6aa admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:56.370090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:56.370829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:56.371185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:56.371528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:56.373276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:56.376365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:56.433936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 803/1606] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:56 2022] POST /identity/v3/projects => generated 355 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:56.434864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:56.436422 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:56.451484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795521f-71e6-414a-b0d4-9871515cfd7e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:56.492943 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:56.530928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:56.531741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:56.532824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:56.535205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:56.536468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:56.540470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:56.611131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:56.623380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 804/1607] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:56 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:56.632926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36ab41-6d7a-4a09-aaee-5e5e72864b75 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:56.648291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:56.648922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:56.649201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:56.649473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:56.650121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:56.652345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:56.682547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 804/1608] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:56.686774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:56.719924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc73480-8cd5-4f79-abcd-c212719a3298 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:56.729673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:56.730687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:56.731050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:56.731223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] PATH_INFO: `/v3/projects/8dc3e2c27de24faaa6035797199ba7e3/users/f178f5cfe0ae4034b9c312c1410d6f47/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:56.757887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc3e2c27de24faaa6035797199ba7e3, user_id=f178f5cfe0ae4034b9c312c1410d6f47, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:56.761957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:56.788070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 805/1609] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:56 2022] PUT /identity/v3/projects/8dc3e2c27de24faaa6035797199ba7e3/users/f178f5cfe0ae4034b9c312c1410d6f47/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:56.793020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:56.823421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1bd63-0d6d-41a4-ad60-28637f040bc9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:56.858575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:56.927253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:56.928009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:56.928684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:56.928684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:56.929415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:56.931122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:56.948278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 805/1610] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:56.950302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:56.961774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f685ed-746c-4d05-a618-a6bd4a92a738 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:56.975456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3bbdb-e906-408d-977b-a30bd273d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:56.986766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3bbdb-e906-408d-977b-a30bd273d868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:56.986766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3bbdb-e906-408d-977b-a30bd273d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:56.986766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3bbdb-e906-408d-977b-a30bd273d868 admin admin] PATH_INFO: `/v3/projects/8dc3e2c27de24faaa6035797199ba7e3/users/f178f5cfe0ae4034b9c312c1410d6f47/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:56.988217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3bbdb-e906-408d-977b-a30bd273d868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc3e2c27de24faaa6035797199ba7e3, user_id=f178f5cfe0ae4034b9c312c1410d6f47, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:56.989704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:56.991538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3bbdb-e906-408d-977b-a30bd273d868 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:56.991797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.008306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 806/1611] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:56 2022] PUT /identity/v3/projects/8dc3e2c27de24faaa6035797199ba7e3/users/f178f5cfe0ae4034b9c312c1410d6f47/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:57.010591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a04e81-f352-4f90-b454-74afbc4d7fc5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:57.026253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.026971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.027244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.085576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] MFA Rules not processed for user `f178f5cfe0ae4034b9c312c1410d6f47`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:57.087104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.160528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:57.161406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.177574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.177998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] PATH_INFO: `/v3/projects/4b67f7c014a6479d9b093ff5d4142fc8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.180460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b67f7c014a6479d9b093ff5d4142fc8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:57.186184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:57.260465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 807/1612] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:57.261452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:57.262872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.286734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b67f7c014a6479d9b093ff5d4142fc8', 'request_id': 'req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.297625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b67f7c014a6479d9b093ff5d4142fc8', 'request_id': 'req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.298759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68addac8-ada3-4b6a-a40d-918b883bf90f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:57.312426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b67f7c014a6479d9b093ff5d4142fc8', 'request_id': 'req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.328519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] The token cache is being invalidate because project 4b67f7c014a6479d9b093ff5d4142fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:57.328857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.329787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.332050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 806/1613] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:56 2022] DELETE /identity/v3/projects/4b67f7c014a6479d9b093ff5d4142fc8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:57.334227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:57.336073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.419073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:57.424812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.425130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.425357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] PATH_INFO: `/v3/projects/b25a94563eb14d2f82cec64fe65d15a5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.427882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25a94563eb14d2f82cec64fe65d15a5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:57.429765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:57.429906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:57.441526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45a11-fb71-4e35-8b8f-0f21f5d6d237 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:57.453271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fb404-b0b4-4b82-9bf5-b7ce4206cfae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:57.454260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fb404-b0b4-4b82-9bf5-b7ce4206cfae service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.454739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fb404-b0b4-4b82-9bf5-b7ce4206cfae service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.455233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fb404-b0b4-4b82-9bf5-b7ce4206cfae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.455761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:57.460496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-474fb404-b0b4-4b82-9bf5-b7ce4206cfae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.549454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25a94563eb14d2f82cec64fe65d15a5', 'request_id': 'req-65190606-b081-4deb-a6c9-bb0a72f60450'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.574540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25a94563eb14d2f82cec64fe65d15a5', 'request_id': 'req-65190606-b081-4deb-a6c9-bb0a72f60450'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.585267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25a94563eb14d2f82cec64fe65d15a5', 'request_id': 'req-65190606-b081-4deb-a6c9-bb0a72f60450'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.605641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fb404-b0b4-4b82-9bf5-b7ce4206cfae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:57.607179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fb404-b0b4-4b82-9bf5-b7ce4206cfae service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:57.625843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] The token cache is being invalidate because project b25a94563eb14d2f82cec64fe65d15a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:24:57.626467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.627911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65190606-b081-4deb-a6c9-bb0a72f60450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:24:57.631383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 808/1614] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:24:57 2022] DELETE /identity/v3/projects/b25a94563eb14d2f82cec64fe65d15a5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:57.634176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.635600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.636510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.645536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 807/1615] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:57.647854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b75e3-d0f7-4c03-9fb1-aff53f3fa81f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.648143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b75e3-d0f7-4c03-9fb1-aff53f3fa81f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.650613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b75e3-d0f7-4c03-9fb1-aff53f3fa81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.660378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:57.661933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.707195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ab2b75e3-d0f7-4c03-9fb1-aff53f3fa81f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:57.708817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ab2b75e3-d0f7-4c03-9fb1-aff53f3fa81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.800984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 809/1616] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:57.802130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:57.803420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.839568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 808/1617] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:57.841113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5ad25-acf3-4da0-9382-2452691608f5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.841397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5ad25-acf3-4da0-9382-2452691608f5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.841708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5ad25-acf3-4da0-9382-2452691608f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.871993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dea5ad25-acf3-4da0-9382-2452691608f5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:57.873525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dea5ad25-acf3-4da0-9382-2452691608f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.882189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 809/1618] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:57.883460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157046-a115-4f21-bd84-87a0fc0ecd58 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.883736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157046-a115-4f21-bd84-87a0fc0ecd58 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.884001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157046-a115-4f21-bd84-87a0fc0ecd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.929937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ce157046-a115-4f21-bd84-87a0fc0ecd58 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:57.931399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eb5d1-de04-4827-933f-c112c15386df None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:57.932841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ce157046-a115-4f21-bd84-87a0fc0ecd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.935830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:57.936658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.937082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.937297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.939170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:57.945928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 810/1619] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:57.947064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157046-a115-4f21-bd84-87a0fc0ecd58 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:57.959910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157046-a115-4f21-bd84-87a0fc0ecd58 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:57.972000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:57.972674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:57.973237 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:57.973511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:57.974159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:57.974457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:57.976031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:57.992896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 811/1620] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:57.992896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.008002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a9ba7-b0f6-4d3d-88f0-d6bbb97d9cd6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.013876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.014498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.014778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.015051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.015698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:24:58.015999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.017624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.017754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.019267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.023316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:58.032638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 810/1621] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:58.033641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.044459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 812/1622] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:24:58.046977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.053591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06f4e3-996e-4c63-af05-f58a43e875ce service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.060023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98815ded-f5ef-42f4-a8c9-aea45a245a82 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.066389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.067166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.067346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.067601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.068269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.070222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.074604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.075235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.075513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.075780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.076452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.078380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.132651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 811/1623] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:58.137089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 813/1624] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:58.137852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.141391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.224421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:58.271376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c4ea3-381a-4cd3-b816-ae6219f7aacd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.284798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775688ce-9737-4063-8dab-848946872deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.285443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775688ce-9737-4063-8dab-848946872deb admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.285722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775688ce-9737-4063-8dab-848946872deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.285997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775688ce-9737-4063-8dab-848946872deb admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.286642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775688ce-9737-4063-8dab-848946872deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.294005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775688ce-9737-4063-8dab-848946872deb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.296182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7952e-2c75-48ea-8ab4-d0358fa6a373 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.302485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.303164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.303437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.303702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.304398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.306676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.339074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 812/1625] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:58.340447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.345930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 814/1626] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:58.346279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775688ce-9737-4063-8dab-848946872deb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.352408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004265b-cdae-4034-9c90-a9fce7852ef1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.358965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.360023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.360332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.360604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.361605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.363961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.366557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775688ce-9737-4063-8dab-848946872deb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.371451 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.372076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.372385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.372657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.373309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.374892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.379624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 813/1627] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:58.381302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.390472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 815/1628] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:24:58.391374 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.394339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec5e7e-701d-4ec8-83ba-10abed7614dc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.399619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.401112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.401390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.401655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.402292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.403897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.405658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6b6b2-09c8-4894-bce8-58c27d2e8d9a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.411219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.412099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.412606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.413077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.414058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.415202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 814/1629] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:58.416130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.419660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.428189 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 816/1630] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:58.432927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.436270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a8395-0df2-44be-babb-487d4de47b70 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.446156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac58a1b-5284-40d4-9298-aff6debea19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.448431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac58a1b-5284-40d4-9298-aff6debea19b admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.449095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac58a1b-5284-40d4-9298-aff6debea19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.449515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2470a-b1ba-4891-aaa6-ffa2995d01e0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.449895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac58a1b-5284-40d4-9298-aff6debea19b admin admin] PATH_INFO: `/v3/projects/c1b1bd1dcec34af9bd5586935f257de2/users/eaca6410a8f546cdb95aecef77dbe551/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.455684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb81d7-454e-4323-a5bf-c275883bc617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.456371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb81d7-454e-4323-a5bf-c275883bc617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.456649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb81d7-454e-4323-a5bf-c275883bc617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.456981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb81d7-454e-4323-a5bf-c275883bc617 admin admin] PATH_INFO: `/v3/projects/b3e93dba96944bafb7a9108403005465/users/32884965929e495ba375b2d13c88ef04/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.477628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac58a1b-5284-40d4-9298-aff6debea19b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b1bd1dcec34af9bd5586935f257de2, user_id=eaca6410a8f546cdb95aecef77dbe551, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.482226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac58a1b-5284-40d4-9298-aff6debea19b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.483835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb81d7-454e-4323-a5bf-c275883bc617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e93dba96944bafb7a9108403005465, user_id=32884965929e495ba375b2d13c88ef04, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.486672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb81d7-454e-4323-a5bf-c275883bc617 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.499573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 815/1631] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:58 2022] PUT /identity/v3/projects/c1b1bd1dcec34af9bd5586935f257de2/users/eaca6410a8f546cdb95aecef77dbe551/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:58.512079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32568e-03c5-46b1-958c-d1cd579580e7 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.512406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32568e-03c5-46b1-958c-d1cd579580e7 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.512684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32568e-03c5-46b1-958c-d1cd579580e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.514502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 817/1632] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:58 2022] PUT /identity/v3/projects/b3e93dba96944bafb7a9108403005465/users/32884965929e495ba375b2d13c88ef04/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:24:58.520436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498088-6828-46a9-bc30-211b152b7604 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.520752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498088-6828-46a9-bc30-211b152b7604 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.521118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498088-6828-46a9-bc30-211b152b7604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.550702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9e32568e-03c5-46b1-958c-d1cd579580e7 None None] MFA Rules not processed for user `eaca6410a8f546cdb95aecef77dbe551`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:58.551966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9e32568e-03c5-46b1-958c-d1cd579580e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:58.555736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-18498088-6828-46a9-bc30-211b152b7604 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:58.557041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-18498088-6828-46a9-bc30-211b152b7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:58.638033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 818/1633] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:24:58.648331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 816/1634] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:58.650548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498088-6828-46a9-bc30-211b152b7604 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.652121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-18498088-6828-46a9-bc30-211b152b7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:58.652799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32568e-03c5-46b1-958c-d1cd579580e7 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.667904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32568e-03c5-46b1-958c-d1cd579580e7 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.673365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.673978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.674933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.674933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.675144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.677121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.685400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:58.719609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 817/1635] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:58.720715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.738958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498088-6828-46a9-bc30-211b152b7604 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.799904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cde9b7-c51b-4dd2-ae1d-9e22cb99225d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.806857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.807554 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.807924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.808294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.808991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.810349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:58.811139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.838778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.839480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.839758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.840026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.851006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 818/1636] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:58 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:58.852312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.858459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.861534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.870289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43cd0-96a1-45a9-8f90-c79c106d3a2b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.873591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 819/1637] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:24:58.875339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.875974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.876299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.876571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.877207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.878863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.893967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 819/1638] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:58.896187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.915212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21996991-b66e-4287-91e6-140b3a3399ff service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.934205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3249242-2c9f-4afb-b969-33280e926c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.944606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3249242-2c9f-4afb-b969-33280e926c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.944606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3249242-2c9f-4afb-b969-33280e926c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.944606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3249242-2c9f-4afb-b969-33280e926c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:58.944606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3249242-2c9f-4afb-b969-33280e926c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:58.944606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3249242-2c9f-4afb-b969-33280e926c04 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:58.967733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 820/1639] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:58.971646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:58.983769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cc110-6b8b-4152-a5e3-0a557cb84cf7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:58.989613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:58.990499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:58.990672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:58.991149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] PATH_INFO: `/v3/projects/c40766db535a4e3c8112450a21bce2f7/users/6d7a51ade5da48639ca395eca6cf8c8d/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.044439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40766db535a4e3c8112450a21bce2f7, user_id=6d7a51ade5da48639ca395eca6cf8c8d, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:59.048045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:59.094372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46620374-e3ab-4caf-8bb2-e73adb479dd1 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.094642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46620374-e3ab-4caf-8bb2-e73adb479dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.094914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46620374-e3ab-4caf-8bb2-e73adb479dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.095841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 820/1640] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:58 2022] PUT /identity/v3/projects/c40766db535a4e3c8112450a21bce2f7/users/6d7a51ade5da48639ca395eca6cf8c8d/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:59.153807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-46620374-e3ab-4caf-8bb2-e73adb479dd1 None None] MFA Rules not processed for user `6d7a51ade5da48639ca395eca6cf8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:24:59.155672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-46620374-e3ab-4caf-8bb2-e73adb479dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:24:59.251440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 821/1641] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:24:59.256175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:59.372872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d080232-85cb-47a0-b267-07669d811a3c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:59.383308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:59.384219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.385244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.385663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.386520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:59.388637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:59.432684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 821/1642] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:24:59 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:24:59.435407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46620374-e3ab-4caf-8bb2-e73adb479dd1 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:59.462145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46620374-e3ab-4caf-8bb2-e73adb479dd1 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:59.620900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:59.621382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.621741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.622021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.624956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:59.629296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:59.665272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 822/1643] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:24:59 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:24:59.666237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:59.677094 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db961b0-1d95-4c9f-8da0-8fcfb2e2a870 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:59.683057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:59.683877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.684369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.684865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.685730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:59.687568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:59.701290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 822/1644] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:24:59.705864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:59.716370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8f425-929a-4e66-88c6-598293c4a592 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:59.721149 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:59.721836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.722117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.722392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] PATH_INFO: `/v3/projects/9000f3d8387b4b9797a0de14a0c1cfde/users/2a72ed3e0dce48848ed4cd4779561f40/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.745067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9000f3d8387b4b9797a0de14a0c1cfde, user_id=2a72ed3e0dce48848ed4cd4779561f40, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:59.745802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:59.770088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 823/1645] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:59 2022] PUT /identity/v3/projects/9000f3d8387b4b9797a0de14a0c1cfde/users/2a72ed3e0dce48848ed4cd4779561f40/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:59.771332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:59.782929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3e234-7134-4eac-86c4-ab6db56fb164 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:59.797116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:24:59.902571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60dc5-673a-4839-b5c8-a711e445b50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:59.903203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60dc5-673a-4839-b5c8-a711e445b50c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.903486 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60dc5-673a-4839-b5c8-a711e445b50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.903758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60dc5-673a-4839-b5c8-a711e445b50c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.904717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d60dc5-673a-4839-b5c8-a711e445b50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:59.909937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d60dc5-673a-4839-b5c8-a711e445b50c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:59.929865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 823/1646] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:24:59.934812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:24:59.945700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd8fe0-2647-406c-a434-d3bf6aa35ed1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:24:59.953470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:24:59.953470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.953470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.953470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] PATH_INFO: `/v3/projects/9000f3d8387b4b9797a0de14a0c1cfde/users/2a72ed3e0dce48848ed4cd4779561f40/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:24:59.954961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9000f3d8387b4b9797a0de14a0c1cfde, user_id=2a72ed3e0dce48848ed4cd4779561f40, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:24:59.958162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:24:59.978953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 824/1647] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:24:59 2022] PUT /identity/v3/projects/9000f3d8387b4b9797a0de14a0c1cfde/users/2a72ed3e0dce48848ed4cd4779561f40/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:24:59.990555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a248-4950-46e4-a487-a76ad6773998 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:24:59.990555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a248-4950-46e4-a487-a76ad6773998 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:24:59.990555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842a248-4950-46e4-a487-a76ad6773998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:00.022020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7842a248-4950-46e4-a487-a76ad6773998 None None] MFA Rules not processed for user `2a72ed3e0dce48848ed4cd4779561f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:00.022342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7842a248-4950-46e4-a487-a76ad6773998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:00.128483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 824/1648] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:24:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:00.144585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:00.146466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:00.161827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:00.218902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a782c6-69f0-4c88-adbf-1f657ba4e789 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:00.223362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:00.224055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:00.224426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:00.224760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:00.310400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:00.312013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:00.324035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 825/1649] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:00.391464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e7deb-f683-4565-ac4c-e7bf5046c157 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:00.391748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e7deb-f683-4565-ac4c-e7bf5046c157 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:00.392016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e7deb-f683-4565-ac4c-e7bf5046c157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:00.441677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c85e7deb-f683-4565-ac4c-e7bf5046c157 None None] MFA Rules not processed for user `32884965929e495ba375b2d13c88ef04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:00.443347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c85e7deb-f683-4565-ac4c-e7bf5046c157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:00.533814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 825/1650] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:00.538045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:00.644269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c2c5-1566-4529-af65-1d3acc5cff79 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:00.656416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:00.657193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:00.657484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:00.657755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:00.658992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:00.664541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:00.726627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 826/1651] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:00 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:00.731930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e7deb-f683-4565-ac4c-e7bf5046c157 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:00.751853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e7deb-f683-4565-ac4c-e7bf5046c157 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:00.820568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:00.875918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:00.876973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:00.877448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:00.877998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:00.880585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:00.882742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:00.908978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 826/1652] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:00 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:00.912880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:00.925020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed69c-e891-4bad-9f44-5731fdffbc43 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:00.931516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:00.932573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:00.933021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:00.933478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:00.934661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:00.936807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:00.952339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 827/1653] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:00.953996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:00.965361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1920-f1a6-4206-8939-f76b4a515fd5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:00.970467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:00.971263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:00.971611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:00.971943 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] PATH_INFO: `/v3/projects/8a8b382c6ef34fda8b176a51909be691/users/c6111fce206b4c199b048707441e1269/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:00.988692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8b382c6ef34fda8b176a51909be691, user_id=c6111fce206b4c199b048707441e1269, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:00.991411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:01.014832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 827/1654] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:00 2022] PUT /identity/v3/projects/8a8b382c6ef34fda8b176a51909be691/users/c6111fce206b4c199b048707441e1269/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:01.018586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:01.057999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb841-1ba6-44d1-84b7-3b7f0a6954af admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:01.171569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:01.240850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:01.241472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:01.241769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:01.242004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:01.242633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:01.245014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:01.255616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 828/1655] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:01.260780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:01.271814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf338c6-c9a6-4e4c-93f4-d55ce17e8159 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:01.276968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4865e-b085-4361-8b37-7d4b10a58d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:01.277741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4865e-b085-4361-8b37-7d4b10a58d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:01.277991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4865e-b085-4361-8b37-7d4b10a58d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:01.278600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4865e-b085-4361-8b37-7d4b10a58d79 admin admin] PATH_INFO: `/v3/projects/8a8b382c6ef34fda8b176a51909be691/users/c6111fce206b4c199b048707441e1269/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:01.280656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4865e-b085-4361-8b37-7d4b10a58d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8b382c6ef34fda8b176a51909be691, user_id=c6111fce206b4c199b048707441e1269, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:01.283342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4865e-b085-4361-8b37-7d4b10a58d79 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:01.293997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 828/1656] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:01 2022] PUT /identity/v3/projects/8a8b382c6ef34fda8b176a51909be691/users/c6111fce206b4c199b048707441e1269/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:02.070281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:02.139967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce30932-c670-4e90-9340-af3d28078e4a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:02.207832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:02.207832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:02.208390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:02.208582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:02.286646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5da47-a916-4ed3-ba4e-28781cef8b88 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:02.287484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5da47-a916-4ed3-ba4e-28781cef8b88 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:02.288499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5da47-a916-4ed3-ba4e-28781cef8b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:02.305720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:02.307190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:02.333624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 829/1657] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:02.349613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b6c5da47-a916-4ed3-ba4e-28781cef8b88 None None] MFA Rules not processed for user `c6111fce206b4c199b048707441e1269`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:02.352645 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b6c5da47-a916-4ed3-ba4e-28781cef8b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:02.501515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 829/1658] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:02.533186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:02.549074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678648e0-8908-4011-8b3e-dcb85a35e213 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:02.563622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:02.564412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:02.564693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:02.564995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:02.627226 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:02.647719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:02.649647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:02.675741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 830/1659] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:03.171132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5da47-a916-4ed3-ba4e-28781cef8b88 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:03.180304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5da47-a916-4ed3-ba4e-28781cef8b88 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:03.184047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:03.184676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:03.184953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:03.185220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:03.254653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:03.256080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:03.260848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:03.268676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 830/1660] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:04.165079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:04.174144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab91b9-f8bc-46b2-a997-9e3b47dad787 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:04.177713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078bec3-66c2-4fab-974b-f7268c328de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:04.178301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078bec3-66c2-4fab-974b-f7268c328de5 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:04.178578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078bec3-66c2-4fab-974b-f7268c328de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:04.178847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078bec3-66c2-4fab-974b-f7268c328de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:04.187594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078bec3-66c2-4fab-974b-f7268c328de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:04.189035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078bec3-66c2-4fab-974b-f7268c328de5 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:04.198458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 831/1661] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:07.576628 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:07.645250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d046a6c-0098-4538-861d-605b0baa3eea service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:07.650708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:07.651341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:07.651613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:07.651891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] PATH_INFO: `/v3/users/eaca6410a8f546cdb95aecef77dbe551` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:07.653077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaca6410a8f546cdb95aecef77dbe551)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:07.655117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:07.716435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaca6410a8f546cdb95aecef77dbe551'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:07.732208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaca6410a8f546cdb95aecef77dbe551'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:07.733330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaca6410a8f546cdb95aecef77dbe551'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:07.741391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaca6410a8f546cdb95aecef77dbe551'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:07.751654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaca6410a8f546cdb95aecef77dbe551'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:07.762104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 831/1662] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:07 2022] DELETE /identity/v3/users/eaca6410a8f546cdb95aecef77dbe551 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:07.766155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:07.766612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:07.766910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:07.787508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:07.789050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:07.865302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 832/1663] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:07.875670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:07.877187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:07.977982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72b375-cca2-4733-9480-111e4f10cf8c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:07.983393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:07.984757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:07.985230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:07.985689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:08.011480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:08.012878 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:08.025762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 832/1664] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:08.409438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:08.411924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:08.437478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7b7c5-fd01-41a8-abfc-e6287d713f51 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:08.445990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:08.446761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:08.447136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:08.447468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] PATH_INFO: `/v3/projects/c1b1bd1dcec34af9bd5586935f257de2` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:08.449382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b1bd1dcec34af9bd5586935f257de2)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:08.452487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:08.542485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b1bd1dcec34af9bd5586935f257de2', 'request_id': 'req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.549784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b1bd1dcec34af9bd5586935f257de2', 'request_id': 'req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.556142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b1bd1dcec34af9bd5586935f257de2', 'request_id': 'req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.567698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] The token cache is being invalidate because project c1b1bd1dcec34af9bd5586935f257de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:08.567991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.568690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.570172 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 833/1665] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:08 2022] DELETE /identity/v3/projects/c1b1bd1dcec34af9bd5586935f257de2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:08.574168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:08.583637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67938de7-5d63-4f12-bdf1-97f7e3294024 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:08.600766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:08.646911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:08.647540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:08.647843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:08.648112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] PATH_INFO: `/v3/users/6d7a51ade5da48639ca395eca6cf8c8d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:08.649529 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d7a51ade5da48639ca395eca6cf8c8d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:08.652076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:08.699272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d7a51ade5da48639ca395eca6cf8c8d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.711550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d7a51ade5da48639ca395eca6cf8c8d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.712187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d7a51ade5da48639ca395eca6cf8c8d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.719794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d7a51ade5da48639ca395eca6cf8c8d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.732531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d7a51ade5da48639ca395eca6cf8c8d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:08.739539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 833/1666] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:08 2022] DELETE /identity/v3/users/6d7a51ade5da48639ca395eca6cf8c8d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:09.017928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:09.020045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:09.051285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc2248-73bd-4b5f-b0cc-01fb495f4d61 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:09.105714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:09.116091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:09.117041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:09.117933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:09.118464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] PATH_INFO: `/v3/projects/c40766db535a4e3c8112450a21bce2f7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:09.119932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40766db535a4e3c8112450a21bce2f7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:09.123424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:09.188707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40766db535a4e3c8112450a21bce2f7', 'request_id': 'req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.198889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40766db535a4e3c8112450a21bce2f7', 'request_id': 'req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.205315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40766db535a4e3c8112450a21bce2f7', 'request_id': 'req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.224498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] The token cache is being invalidate because project c40766db535a4e3c8112450a21bce2f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:09.224963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.225771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.227836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 834/1667] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:09 2022] DELETE /identity/v3/projects/c40766db535a4e3c8112450a21bce2f7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:09.231758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:09.237272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:09.250476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6ceb4-54aa-4b12-8ede-03250eff284a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:09.311552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:09.312224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:09.312568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:09.312841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] PATH_INFO: `/v3/users/2a72ed3e0dce48848ed4cd4779561f40` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:09.314001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a72ed3e0dce48848ed4cd4779561f40)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:09.315934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:09.363797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:09.370002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a72ed3e0dce48848ed4cd4779561f40'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.382950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a72ed3e0dce48848ed4cd4779561f40'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.383650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a72ed3e0dce48848ed4cd4779561f40'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.391009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a72ed3e0dce48848ed4cd4779561f40'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.399394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-086dae72-b8aa-4e0b-8ca7-c5c5b52cf520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a72ed3e0dce48848ed4cd4779561f40'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:09.405458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 834/1668] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:09 2022] DELETE /identity/v3/users/2a72ed3e0dce48848ed4cd4779561f40 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:10.034185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.036215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:10.051029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3028798-c9ff-459d-8fd8-c9ce0cc2d4b0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.116945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.117582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.117861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.118137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] PATH_INFO: `/v3/projects/9000f3d8387b4b9797a0de14a0c1cfde` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.119851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9000f3d8387b4b9797a0de14a0c1cfde)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.121770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:10.168321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9000f3d8387b4b9797a0de14a0c1cfde', 'request_id': 'req-ea55c76f-846f-44de-8242-d8d3bfcc70aa'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:10.176291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9000f3d8387b4b9797a0de14a0c1cfde', 'request_id': 'req-ea55c76f-846f-44de-8242-d8d3bfcc70aa'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:10.182593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9000f3d8387b4b9797a0de14a0c1cfde', 'request_id': 'req-ea55c76f-846f-44de-8242-d8d3bfcc70aa'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:10.194458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] The token cache is being invalidate because project 9000f3d8387b4b9797a0de14a0c1cfde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:10.194873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:10.195519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ea55c76f-846f-44de-8242-d8d3bfcc70aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:10.197717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 835/1669] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:10 2022] DELETE /identity/v3/projects/9000f3d8387b4b9797a0de14a0c1cfde => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:10.202517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f0da-33e4-488d-8d1f-9a0df2659810 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.202975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f0da-33e4-488d-8d1f-9a0df2659810 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.203157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f0da-33e4-488d-8d1f-9a0df2659810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.222315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-dd00f0da-33e4-488d-8d1f-9a0df2659810 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:10.223493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-dd00f0da-33e4-488d-8d1f-9a0df2659810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:10.290480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 835/1670] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:10.299525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d9e4e-6334-4230-8de2-5edeb2c6c720 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.300457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d9e4e-6334-4230-8de2-5edeb2c6c720 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.300826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d9e4e-6334-4230-8de2-5edeb2c6c720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.332478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c89d9e4e-6334-4230-8de2-5edeb2c6c720 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:10.332478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c89d9e4e-6334-4230-8de2-5edeb2c6c720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:10.335678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 836/1671] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:10.341443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00f0da-33e4-488d-8d1f-9a0df2659810 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.352031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00f0da-33e4-488d-8d1f-9a0df2659810 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.358808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.359423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.359701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.359972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.360844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:10.361150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.363320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:10.380111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 836/1672] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:10.381906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d9e4e-6334-4230-8de2-5edeb2c6c720 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.393274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d9e4e-6334-4230-8de2-5edeb2c6c720 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.399703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.400394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.400672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.400937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.401570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.404012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:10.412819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:10.445731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 837/1673] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:10 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:10.450272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.461433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87354c-c3cf-4180-bc02-42487e3e59a0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.520608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.521225 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.521500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.521769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.522406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.524357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:10.551968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 837/1674] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:10 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:10.553812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.565496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867230c-8157-4c3d-82f7-86017351de05 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.571107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.571936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.572308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.573056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.573340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.575318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:10.588145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 838/1675] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:10.592545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.609622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efdccd-493e-42d8-b02c-4365aff246a6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.622259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98612d5b-dd53-4c07-bec8-613910bb75ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.622556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98612d5b-dd53-4c07-bec8-613910bb75ce admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.622903 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98612d5b-dd53-4c07-bec8-613910bb75ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.623232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98612d5b-dd53-4c07-bec8-613910bb75ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.623951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98612d5b-dd53-4c07-bec8-613910bb75ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.625662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98612d5b-dd53-4c07-bec8-613910bb75ce admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:10.637377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 838/1676] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:10.641717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.652307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69483832-b88c-46a2-9f56-f422b74a9b15 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.658149 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.658897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.659284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.659625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] PATH_INFO: `/v3/projects/cd7624056a1342a19b08c707ea89a0be/users/848e7df435c0452aae343d68746a1132/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.677000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7624056a1342a19b08c707ea89a0be, user_id=848e7df435c0452aae343d68746a1132, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.679720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:10.707482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 839/1677] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:10 2022] PUT /identity/v3/projects/cd7624056a1342a19b08c707ea89a0be/users/848e7df435c0452aae343d68746a1132/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:10.709339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a7937-accd-425e-9901-751bace2a77e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.710351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a7937-accd-425e-9901-751bace2a77e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.710351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a7937-accd-425e-9901-751bace2a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.742144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3e8a7937-accd-425e-9901-751bace2a77e None None] MFA Rules not processed for user `848e7df435c0452aae343d68746a1132`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:10.742860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3e8a7937-accd-425e-9901-751bace2a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:10.758865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:10.822140 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 839/1678] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:10.827030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:10.950296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13aeaa-ae0c-499d-992a-69b7f5fa4c83 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:10.958150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:10.958937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:10.960516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:10.960516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:10.960779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:10.966251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:11.008915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 840/1679] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:10 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:11.013679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a7937-accd-425e-9901-751bace2a77e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:11.029472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a7937-accd-425e-9901-751bace2a77e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:11.153938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:11.159081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:11.159549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:11.161402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:11.162282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:11.165082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:11.240027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 840/1680] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:11 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:11.244595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:11.263433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037fe3f-f8d5-4274-8f19-180337595a51 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:11.287974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:11.288584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:11.289409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:11.289916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:11.291876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:11.294289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:11.337719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:11.337963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 841/1681] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:11.366448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156868b-390d-4821-a165-b10149df9e5f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:11.372688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3021f9-8ec7-40da-9c82-a83d65a470f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:11.373287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3021f9-8ec7-40da-9c82-a83d65a470f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:11.373568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3021f9-8ec7-40da-9c82-a83d65a470f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:11.373878 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3021f9-8ec7-40da-9c82-a83d65a470f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:11.374471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3021f9-8ec7-40da-9c82-a83d65a470f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:11.376429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3021f9-8ec7-40da-9c82-a83d65a470f5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:11.390514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 841/1682] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:11.395113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:11.406509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023d89a-04a5-47cd-89d4-03a7648c38b4 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:11.411878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:11.412583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:11.412864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:11.413135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] PATH_INFO: `/v3/projects/7fce067ad4f54bdfa814eadce2e06111/users/e848173ee2f14b43a1f4fbf95ba600e8/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:11.430390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fce067ad4f54bdfa814eadce2e06111, user_id=e848173ee2f14b43a1f4fbf95ba600e8, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:11.433401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:11.476866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 842/1683] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:11 2022] PUT /identity/v3/projects/7fce067ad4f54bdfa814eadce2e06111/users/e848173ee2f14b43a1f4fbf95ba600e8/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:11.479653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c65e0-2039-4eab-bf71-aa4250bc9028 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:11.480728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c65e0-2039-4eab-bf71-aa4250bc9028 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:11.481422 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c65e0-2039-4eab-bf71-aa4250bc9028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:11.522146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-575c65e0-2039-4eab-bf71-aa4250bc9028 None None] MFA Rules not processed for user `e848173ee2f14b43a1f4fbf95ba600e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:11.524377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-575c65e0-2039-4eab-bf71-aa4250bc9028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:11.699713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 842/1684] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:11.704955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:11.797746 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a750a-873c-4ef6-9748-f9ff321e3985 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:11.811298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:11.811936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:11.812220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:11.812524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:11.813173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:11.820752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:11.829614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:11.877390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 843/1685] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:11 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:11.881531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c65e0-2039-4eab-bf71-aa4250bc9028 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:11.895282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c65e0-2039-4eab-bf71-aa4250bc9028 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:11.917388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:11.974185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:11.974853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:11.975031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:11.975309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:11.975947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:11.978442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:12.032787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 843/1686] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:11 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:12.033558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:12.048642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faa746-3fd8-446f-b80e-57ec6b6ebeca admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:12.061871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:12.062515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:12.062822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:12.063055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:12.063860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:12.065602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:12.080819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 844/1687] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:12.085702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:12.097091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3d54a-3b5a-4426-84d4-3b936983d70b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:12.101784 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:12.103345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:12.103690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:12.104030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] PATH_INFO: `/v3/projects/cc94a4d98e554edba293c85ef8474db8/users/133140bb0b904af4b2cb5330ff6db312/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:12.119694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc94a4d98e554edba293c85ef8474db8, user_id=133140bb0b904af4b2cb5330ff6db312, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:12.122426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:12.136647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 844/1688] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:12 2022] PUT /identity/v3/projects/cc94a4d98e554edba293c85ef8474db8/users/133140bb0b904af4b2cb5330ff6db312/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:12.140681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:12.151654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0528d-6267-4418-934e-6f363b0f0886 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:12.217327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268f414-2a8a-41ca-9d68-795dd7f2fd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:12.217956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268f414-2a8a-41ca-9d68-795dd7f2fd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:12.218236 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268f414-2a8a-41ca-9d68-795dd7f2fd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:12.218508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268f414-2a8a-41ca-9d68-795dd7f2fd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:12.219153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268f414-2a8a-41ca-9d68-795dd7f2fd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:12.220853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268f414-2a8a-41ca-9d68-795dd7f2fd5f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:12.235878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 845/1689] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:12.237110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:12.253782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79630fa-d504-44cf-93a6-85e9f18108b7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:12.259810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:12.260868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:12.261416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:12.261885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] PATH_INFO: `/v3/projects/cc94a4d98e554edba293c85ef8474db8/users/133140bb0b904af4b2cb5330ff6db312/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:12.264443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc94a4d98e554edba293c85ef8474db8, user_id=133140bb0b904af4b2cb5330ff6db312, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:12.267170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:12.279594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 845/1690] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:12 2022] PUT /identity/v3/projects/cc94a4d98e554edba293c85ef8474db8/users/133140bb0b904af4b2cb5330ff6db312/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:12.291471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94b3df-44a1-478f-840e-4192547b5fdf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:12.291957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94b3df-44a1-478f-840e-4192547b5fdf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:12.292039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94b3df-44a1-478f-840e-4192547b5fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:12.322737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-da94b3df-44a1-478f-840e-4192547b5fdf None None] MFA Rules not processed for user `133140bb0b904af4b2cb5330ff6db312`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:12.323874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-da94b3df-44a1-478f-840e-4192547b5fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:12.436894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 846/1691] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:12.450814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:12.452935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:12.541588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5895ae-5d07-45de-aa66-af379b92788d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:12.549156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18fc29-e8eb-4ec8-98eb-5f500043c277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:12.549156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18fc29-e8eb-4ec8-98eb-5f500043c277 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:12.554040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18fc29-e8eb-4ec8-98eb-5f500043c277 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:12.554040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18fc29-e8eb-4ec8-98eb-5f500043c277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:12.664064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18fc29-e8eb-4ec8-98eb-5f500043c277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:12.666408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18fc29-e8eb-4ec8-98eb-5f500043c277 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:12.678517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 846/1692] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:13.756769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d45df-4094-4f9c-9dc4-997b2f2cf257 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:13.757219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d45df-4094-4f9c-9dc4-997b2f2cf257 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:13.757219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d45df-4094-4f9c-9dc4-997b2f2cf257 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:13.758762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 847/1693] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:25:13.773430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:13.773966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:13.774043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:13.810290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:13.812919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:13.863310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d45df-4094-4f9c-9dc4-997b2f2cf257 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:13.865846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-662d45df-4094-4f9c-9dc4-997b2f2cf257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:13.866543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:13.913696 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 847/1694] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:13.931323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:13.932790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:13.946297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497c8e9-c726-42ba-b313-b2a2cf573c94 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:13.951912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:13.952558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:13.952838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:13.953102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:13.955266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:13.963538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:13.965267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:13.965484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d45df-4094-4f9c-9dc4-997b2f2cf257 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:13.970505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:13.971152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:13.971432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:13.971707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] PATH_INFO: `/v3/users/24289ae31c444a03b59ca5322f5abff2` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:13.972872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=24289ae31c444a03b59ca5322f5abff2)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:13.974808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:13.975544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 848/1695] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:14.033619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24289ae31c444a03b59ca5322f5abff2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.049688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24289ae31c444a03b59ca5322f5abff2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.050542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24289ae31c444a03b59ca5322f5abff2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.063766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24289ae31c444a03b59ca5322f5abff2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.072330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ece50937-d01d-45cc-b78f-ba3d6a7bba3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24289ae31c444a03b59ca5322f5abff2'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.088651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 848/1696] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:13 2022] DELETE /identity/v3/users/24289ae31c444a03b59ca5322f5abff2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:14.260546 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.273465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009db37d-3157-4e65-af05-3a73ebabebef service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.334278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.334909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.335182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.335447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] PATH_INFO: `/v3/projects/cecd3a5ba83e47119462cb9166e9a80a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.336578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cecd3a5ba83e47119462cb9166e9a80a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.338999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.390363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cecd3a5ba83e47119462cb9166e9a80a', 'request_id': 'req-893d6ff0-7238-4763-b4f8-a711507ab989'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.397453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cecd3a5ba83e47119462cb9166e9a80a', 'request_id': 'req-893d6ff0-7238-4763-b4f8-a711507ab989'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.403089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cecd3a5ba83e47119462cb9166e9a80a', 'request_id': 'req-893d6ff0-7238-4763-b4f8-a711507ab989'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.414408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] The token cache is being invalidate because project cecd3a5ba83e47119462cb9166e9a80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:14.414697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.415874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:14.417367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 849/1697] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:14 2022] DELETE /identity/v3/projects/cecd3a5ba83e47119462cb9166e9a80a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:14.422225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0eeb0-084e-46d7-b790-e4039cb0c54b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.422504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0eeb0-084e-46d7-b790-e4039cb0c54b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.422772 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0eeb0-084e-46d7-b790-e4039cb0c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.433042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.434393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:14.442880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f6c0eeb0-084e-46d7-b790-e4039cb0c54b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:14.444027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f6c0eeb0-084e-46d7-b790-e4039cb0c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:14.481591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:14.495346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d6ff0-7238-4763-b4f8-a711507ab989 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.499113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.499720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.499991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.500291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.502005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:14.503648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 849/1698] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:14.511871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c3560-cf15-4ed3-b65a-9ecb98b364f6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.512156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c3560-cf15-4ed3-b65a-9ecb98b364f6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.512456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c3560-cf15-4ed3-b65a-9ecb98b364f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.532130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b97c3560-cf15-4ed3-b65a-9ecb98b364f6 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:14.533503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b97c3560-cf15-4ed3-b65a-9ecb98b364f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:14.542517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 850/1699] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:14.546342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c3560-cf15-4ed3-b65a-9ecb98b364f6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.555428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c3560-cf15-4ed3-b65a-9ecb98b364f6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.560171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.560415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.560785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.561061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.561327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.561872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.561970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:14.562241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.563792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.571274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 850/1700] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:14.582663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 851/1701] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:25:14.584084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.600249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27106c-6843-48cb-a8b2-742c279a37ac service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.605511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.606099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.606370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.606634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.607271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.609170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.636363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 851/1702] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:14 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:14.640548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.651966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90c615-7fb4-4982-bbf8-78404b96aadb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.717984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:14.725100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.725705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.725918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.726180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.726806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.729346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.752121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 852/1703] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:14 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:14.756042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.781192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d9c21-46ec-4f2b-8790-8b366b02cbe8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.786609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.787148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.787578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.787991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.788820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.791071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.805908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 852/1704] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:14.807173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.819683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6897210-95c0-4c1e-a232-b1f7188ca5cf admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.825973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca13b-283a-48cc-9394-2b4e1435b951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.826988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca13b-283a-48cc-9394-2b4e1435b951 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.827321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca13b-283a-48cc-9394-2b4e1435b951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.827539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca13b-283a-48cc-9394-2b4e1435b951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.828585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca13b-283a-48cc-9394-2b4e1435b951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.830210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca13b-283a-48cc-9394-2b4e1435b951 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.841860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 853/1705] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:14.847654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.858539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf4c2d-5ba3-48db-ad5f-fe2a0fce395a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:14.863217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:14.863949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.864223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.864529 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] PATH_INFO: `/v3/projects/095dae174c664eafaf3ed463d8b7f319/users/a385ba4ea6ee4a2fa76f0f12719427ee/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.879067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095dae174c664eafaf3ed463d8b7f319, user_id=a385ba4ea6ee4a2fa76f0f12719427ee, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:14.881693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:14.895141 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 853/1706] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:14 2022] PUT /identity/v3/projects/095dae174c664eafaf3ed463d8b7f319/users/a385ba4ea6ee4a2fa76f0f12719427ee/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:14.900794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:14.901073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:14.901339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:14.920519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:14.921704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:14.980636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 854/1707] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:14.990130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:14.991831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:15.049976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145c944-e668-4609-8bb0-4c91d3397d16 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:15.053503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:15.054106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:15.054390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:15.054650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:15.063397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:15.064953 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:15.075002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 854/1708] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:17.349319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:17.350913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:17.435171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8cbf-bf8f-49be-afd9-e393e9e56cf5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:17.438861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa17ce3-a3fb-444a-930f-f16da80c109a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:17.439500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa17ce3-a3fb-444a-930f-f16da80c109a service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:17.439775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa17ce3-a3fb-444a-930f-f16da80c109a service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:17.440264 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa17ce3-a3fb-444a-930f-f16da80c109a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:17.442280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-caa17ce3-a3fb-444a-930f-f16da80c109a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:17.520500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa17ce3-a3fb-444a-930f-f16da80c109a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:17.522086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa17ce3-a3fb-444a-930f-f16da80c109a service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:17.534459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 855/1709] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:18.271194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:18.273113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:18.286970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c24f83-ef9f-4760-92e8-e17286e722ed service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:18.292490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:18.293300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:18.293725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:18.293997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] PATH_INFO: `/v3/users/84215a77e65248099434dc4d9884ba7a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:18.295115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84215a77e65248099434dc4d9884ba7a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:18.297050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:18.358939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:18.387872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84215a77e65248099434dc4d9884ba7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:18.407457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84215a77e65248099434dc4d9884ba7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:18.410711 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84215a77e65248099434dc4d9884ba7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:18.426902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84215a77e65248099434dc4d9884ba7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:18.441623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84215a77e65248099434dc4d9884ba7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:18.460547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 855/1710] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:18 2022] DELETE /identity/v3/users/84215a77e65248099434dc4d9884ba7a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:18.465817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:18.466886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:18.467409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:18.495300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:18.497262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:18.538957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:18.638035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 856/1711] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:18.647770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:18.649998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:18.793228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b0082-0396-4705-98f1-4e95d4063469 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:18.798861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:18.799698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:18.800071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:18.800734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:18.814166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:18.817604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:18.836071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 856/1712] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:19.625750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:19.627885 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:19.641680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76891993-ad35-4f2e-9400-fb152977c1c9 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:19.648547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:19.650203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:19.650664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:19.651167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] PATH_INFO: `/v3/projects/99b150fa597a437a97ed50393637d3e2` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:19.655341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b150fa597a437a97ed50393637d3e2)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:19.659320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:19.809555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b150fa597a437a97ed50393637d3e2', 'request_id': 'req-fe202ac5-5b05-44a7-98b0-efa09fd064bf'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:19.845685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b150fa597a437a97ed50393637d3e2', 'request_id': 'req-fe202ac5-5b05-44a7-98b0-efa09fd064bf'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:19.854358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b150fa597a437a97ed50393637d3e2', 'request_id': 'req-fe202ac5-5b05-44a7-98b0-efa09fd064bf'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:19.868083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] The token cache is being invalidate because project 99b150fa597a437a97ed50393637d3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:19.871014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:19.872444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fe202ac5-5b05-44a7-98b0-efa09fd064bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:19.876691 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 857/1713] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:19 2022] DELETE /identity/v3/projects/99b150fa597a437a97ed50393637d3e2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:19.881099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:19.892778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23493f06-1f6c-47a8-9bbf-b5dabcd8dca6 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:19.916675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:19.916833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:19.917352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:19.960931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] MFA Rules not processed for user `a385ba4ea6ee4a2fa76f0f12719427ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:19.963466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:19.990118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:19.991304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:19.991586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:19.992799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] PATH_INFO: `/v3/users/f178f5cfe0ae4034b9c312c1410d6f47` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:19.994082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f178f5cfe0ae4034b9c312c1410d6f47)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:19.999980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:20.069831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 858/1714] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:20.085632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:20.086665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f178f5cfe0ae4034b9c312c1410d6f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.087091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:20.101060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f178f5cfe0ae4034b9c312c1410d6f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.101967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f178f5cfe0ae4034b9c312c1410d6f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.105564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:20.114086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f178f5cfe0ae4034b9c312c1410d6f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.125979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f178f5cfe0ae4034b9c312c1410d6f47'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.134919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 857/1715] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:19 2022] DELETE /identity/v3/users/f178f5cfe0ae4034b9c312c1410d6f47 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:20.140888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:20.199028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834316c9-d7bf-4210-bb5c-ed70c0b29b71 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:20.291524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126cc7b-cca0-4ebe-b942-94087f9e25cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:20.292298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126cc7b-cca0-4ebe-b942-94087f9e25cf service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:20.292530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126cc7b-cca0-4ebe-b942-94087f9e25cf service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:20.292800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126cc7b-cca0-4ebe-b942-94087f9e25cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:20.294673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2126cc7b-cca0-4ebe-b942-94087f9e25cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:20.370510 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126cc7b-cca0-4ebe-b942-94087f9e25cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:20.372127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126cc7b-cca0-4ebe-b942-94087f9e25cf service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:20.387424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 859/1716] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:20.537236 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:20.538673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:20.669044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcecad6-6034-4acd-92cd-5c80e2067697 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:20.687837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:20.688512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:20.689089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:20.689737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] PATH_INFO: `/v3/projects/8dc3e2c27de24faaa6035797199ba7e3` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:20.691723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc3e2c27de24faaa6035797199ba7e3)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:20.693486 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:20.808704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc3e2c27de24faaa6035797199ba7e3', 'request_id': 'req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.825872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc3e2c27de24faaa6035797199ba7e3', 'request_id': 'req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.832313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc3e2c27de24faaa6035797199ba7e3', 'request_id': 'req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.849344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] The token cache is being invalidate because project 8dc3e2c27de24faaa6035797199ba7e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:20.849660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.850310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa39b590-d1a1-4a84-9d7c-2e2a96bbedda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:20.851816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 858/1717] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:20 2022] DELETE /identity/v3/projects/8dc3e2c27de24faaa6035797199ba7e3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:20.857491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858b0e7-78cc-4823-9014-9bdf90553c14 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:20.858629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858b0e7-78cc-4823-9014-9bdf90553c14 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:20.858925 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858b0e7-78cc-4823-9014-9bdf90553c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:20.902114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a858b0e7-78cc-4823-9014-9bdf90553c14 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:20.903555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a858b0e7-78cc-4823-9014-9bdf90553c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:21.036313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5772539-0adc-4330-b0de-63db862c7c13 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.037513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5772539-0adc-4330-b0de-63db862c7c13 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.037936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5772539-0adc-4330-b0de-63db862c7c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.047320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 860/1718] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:21.084181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b5772539-0adc-4330-b0de-63db862c7c13 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:21.087678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b5772539-0adc-4330-b0de-63db862c7c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:21.097947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 859/1719] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:21.103156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858b0e7-78cc-4823-9014-9bdf90553c14 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:21.130280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858b0e7-78cc-4823-9014-9bdf90553c14 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:21.137951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:21.138857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.139208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.139553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.140375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:21.140777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:21.142455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:21.172777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 861/1720] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:21.173793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5772539-0adc-4330-b0de-63db862c7c13 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:21.221157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5772539-0adc-4330-b0de-63db862c7c13 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:21.235610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:21.237165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.238298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.239237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.241682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:21.247530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:21.290636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 860/1721] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:21 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:21.295234 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:21.304719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:21.315582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fee911-5b66-43d3-97bc-ad43b886c575 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:21.462009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:21.462955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.463434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.463902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.465055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:21.467304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:21.514719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 862/1722] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:21 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:21.517475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:21.533530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab9f75-e8d1-43de-a8b8-38facbde41a4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:21.552948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:21.553585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.553869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.554146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.554795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:21.557004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:21.583413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 861/1723] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:21.587444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:21.610577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eed50f-1070-4a0b-98da-fc5160b42ab5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:21.625104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e20ee-1d42-438b-b9a5-009ecfd26d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:21.625734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e20ee-1d42-438b-b9a5-009ecfd26d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.626014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e20ee-1d42-438b-b9a5-009ecfd26d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.626287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e20ee-1d42-438b-b9a5-009ecfd26d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.627347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e20ee-1d42-438b-b9a5-009ecfd26d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:21.629103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e20ee-1d42-438b-b9a5-009ecfd26d5f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:21.643863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 863/1724] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:21.648422 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:21.661779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6817-5b93-4595-9394-ec35355be0d8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:21.668263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:21.669323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.669713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.670052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] PATH_INFO: `/v3/projects/5d1f61d01c3042d78cca1b099e24691c/users/6ac1c9bbf1bb452fb7cc72c59857ea9e/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.691698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1f61d01c3042d78cca1b099e24691c, user_id=6ac1c9bbf1bb452fb7cc72c59857ea9e, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:21.694553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:21.726308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 862/1725] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:21 2022] PUT /identity/v3/projects/5d1f61d01c3042d78cca1b099e24691c/users/6ac1c9bbf1bb452fb7cc72c59857ea9e/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:21.737629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae9af5-31fd-4da3-882a-ac710f19d71c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.737911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae9af5-31fd-4da3-882a-ac710f19d71c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.738180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae9af5-31fd-4da3-882a-ac710f19d71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.785468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b7ae9af5-31fd-4da3-882a-ac710f19d71c None None] MFA Rules not processed for user `6ac1c9bbf1bb452fb7cc72c59857ea9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:21.786692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b7ae9af5-31fd-4da3-882a-ac710f19d71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:21.880703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 864/1726] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:21.881907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:21.888064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:21.963134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cbdec-5f70-444b-a701-5dfe0e2d97d5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:21.967927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:21.968573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:21.968851 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:21.969121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:21.970021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:21.971910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.004510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 863/1727] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:21 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:22.005865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae9af5-31fd-4da3-882a-ac710f19d71c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.017064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae9af5-31fd-4da3-882a-ac710f19d71c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.076395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.076923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.077198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.077463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.078088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.079980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.096930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:22.105631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 865/1728] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:22 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:22.107058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.119490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3000-3527-4307-bcd0-981679dd36a8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.126429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.127508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.127799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.128076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.128788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.130394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.144839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 864/1729] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:22.148487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.166383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50a8c-97e0-4337-962c-f3215b906676 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.171636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.172845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.173121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.173387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] PATH_INFO: `/v3/projects/4dfe920f9498418bada826fc59afdb91/users/6b61e32b0a4d4dd19b8ab8560e30e43f/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.192676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dfe920f9498418bada826fc59afdb91, user_id=6b61e32b0a4d4dd19b8ab8560e30e43f, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.195395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.212925 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 866/1730] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:22 2022] PUT /identity/v3/projects/4dfe920f9498418bada826fc59afdb91/users/6b61e32b0a4d4dd19b8ab8560e30e43f/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:22.216624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.223052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.224912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:22.228137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03ee33-5a45-4313-9cb3-bcf2a3fe1a1b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.308742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8c05-46a2-4b9c-a6ca-d08f1be67680 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.314257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.315002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.315650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.316029 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.317218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.318216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.319200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.319314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.319687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.320072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] PATH_INFO: `/v3/users/848e7df435c0452aae343d68746a1132` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.323185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=848e7df435c0452aae343d68746a1132)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.325520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.336261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 865/1731] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:22.340305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.353470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb56ca-371b-43d7-b9cb-62c93506d439 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.360804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c744b60-d445-4c80-b53b-be48fcf0d418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.361544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c744b60-d445-4c80-b53b-be48fcf0d418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.361967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c744b60-d445-4c80-b53b-be48fcf0d418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.362383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c744b60-d445-4c80-b53b-be48fcf0d418 admin admin] PATH_INFO: `/v3/projects/4dfe920f9498418bada826fc59afdb91/users/6b61e32b0a4d4dd19b8ab8560e30e43f/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.364705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c744b60-d445-4c80-b53b-be48fcf0d418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dfe920f9498418bada826fc59afdb91, user_id=6b61e32b0a4d4dd19b8ab8560e30e43f, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.367334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c744b60-d445-4c80-b53b-be48fcf0d418 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.387518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 866/1732] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:22 2022] PUT /identity/v3/projects/4dfe920f9498418bada826fc59afdb91/users/6b61e32b0a4d4dd19b8ab8560e30e43f/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:22.398048 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.398430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.398676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.440034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] MFA Rules not processed for user `6b61e32b0a4d4dd19b8ab8560e30e43f`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:22.440969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848e7df435c0452aae343d68746a1132'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.441301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:22.458497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848e7df435c0452aae343d68746a1132'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.460683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848e7df435c0452aae343d68746a1132'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.470723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848e7df435c0452aae343d68746a1132'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.481459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-97fbc976-e136-4720-8bf4-038c7c2986b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848e7df435c0452aae343d68746a1132'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.490339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 867/1733] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:22 2022] DELETE /identity/v3/users/848e7df435c0452aae343d68746a1132 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:22.496783 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.497086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.497344 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.511867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:22.520855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:22.522118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:22.537214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 867/1734] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:22.553189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.554700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:22.597488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:22.618117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 868/1735] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:22.630896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.632339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:22.724438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae0854-08a8-4434-862b-50d291f6da15 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.741994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071433a-3308-49f9-ae2b-201f6d294123 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.746947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.747745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.748140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.749167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.764924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.767597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.781843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 869/1736] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:22.794069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.794817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.795179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.795520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.816105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:22.818767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:22.836456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28aa8a-6831-45e3-9d1d-43c4a53c5e62 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:22.843794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:22.844525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:22.844971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:22.845243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] PATH_INFO: `/v3/users/32884965929e495ba375b2d13c88ef04` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:22.846428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32884965929e495ba375b2d13c88ef04)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.848812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.912382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:22.912382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:22.922755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 868/1737] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:22.963865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32884965929e495ba375b2d13c88ef04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.980707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32884965929e495ba375b2d13c88ef04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.981769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32884965929e495ba375b2d13c88ef04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:22.990926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32884965929e495ba375b2d13c88ef04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.005670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32884965929e495ba375b2d13c88ef04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.015828 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 870/1738] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:22 2022] DELETE /identity/v3/users/32884965929e495ba375b2d13c88ef04 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:23.240117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:23.257772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c11daf-2db2-4eab-83d3-7428f0a62610 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:23.334280 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:23.354052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:23.354987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:23.355548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:23.356180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] PATH_INFO: `/v3/projects/b3e93dba96944bafb7a9108403005465` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:23.357967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e93dba96944bafb7a9108403005465)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:23.360852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:23.441524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:23.443104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:23.450417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e93dba96944bafb7a9108403005465', 'request_id': 'req-e4168c20-709f-4dce-9871-e149ecee8f20'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.474518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e93dba96944bafb7a9108403005465', 'request_id': 'req-e4168c20-709f-4dce-9871-e149ecee8f20'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.485396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e93dba96944bafb7a9108403005465', 'request_id': 'req-e4168c20-709f-4dce-9871-e149ecee8f20'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.516198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] The token cache is being invalidate because project b3e93dba96944bafb7a9108403005465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:23.520205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.520788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.527261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 869/1739] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:23 2022] DELETE /identity/v3/projects/b3e93dba96944bafb7a9108403005465 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:23.528600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:23.532373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:23.567396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359b024-3114-4c6c-ae4f-4dd305f7993b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:23.583140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:23.643356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4168c20-709f-4dce-9871-e149ecee8f20 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:23.651839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:23.653240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:23.653644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:23.654011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] PATH_INFO: `/v3/users/c6111fce206b4c199b048707441e1269` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:23.655333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6111fce206b4c199b048707441e1269)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:23.657713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:23.667814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2b7d5-3f06-482d-8a4a-e22bcea4b355 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b61e32b0a4d4dd19b8ab8560e30e43f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4dfe920f9498418bada826fc59afdb91', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:23.668470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2b7d5-3f06-482d-8a4a-e22bcea4b355 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:23.668942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2b7d5-3f06-482d-8a4a-e22bcea4b355 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:23.669216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2b7d5-3f06-482d-8a4a-e22bcea4b355 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:23.669986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2b7d5-3f06-482d-8a4a-e22bcea4b355 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:23.670265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2b7d5-3f06-482d-8a4a-e22bcea4b355 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:23.672268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2b7d5-3f06-482d-8a4a-e22bcea4b355 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:23.689791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 871/1740] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:23.690618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:23.690909 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:23.691355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:23.719547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] MFA Rules not processed for user `a385ba4ea6ee4a2fa76f0f12719427ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:23.720918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:23.748741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6111fce206b4c199b048707441e1269'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.771217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6111fce206b4c199b048707441e1269'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.771982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6111fce206b4c199b048707441e1269'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.785924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6111fce206b4c199b048707441e1269'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.794490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6111fce206b4c199b048707441e1269'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:23.797257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:23.802136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 870/1741] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:23 2022] DELETE /identity/v3/users/c6111fce206b4c199b048707441e1269 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:23.803017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:23.807142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:23.813097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 872/1742] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:23.815039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:23.816734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:23.832478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcfd8d-6d8b-4708-840c-259d36cc3696 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:23.928179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b61e32b0a4d4dd19b8ab8560e30e43f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4dfe920f9498418bada826fc59afdb91', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:23.928842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:23.929100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:23.929477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:23.930966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:23.933014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:23.941264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d288e6-b6db-4843-809e-72326c93fe4a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:23.946769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:23.947557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:23.947838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:23.948112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] PATH_INFO: `/v3/projects/cd7624056a1342a19b08c707ea89a0be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:23.949493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7624056a1342a19b08c707ea89a0be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:23.951994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:23.969541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 873/1743] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:23 2022] POST /identity/v3/projects => generated 353 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:23.974224 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:23.979367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:24.045651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7624056a1342a19b08c707ea89a0be', 'request_id': 'req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.063143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:24.073221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7624056a1342a19b08c707ea89a0be', 'request_id': 'req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.087140 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7624056a1342a19b08c707ea89a0be', 'request_id': 'req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.114877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbed7c2-91cc-412a-8e9e-3e44d70d34dd tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:24.119308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] The token cache is being invalidate because project cd7624056a1342a19b08c707ea89a0be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:24.119603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.122891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.130442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 871/1744] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:23 2022] DELETE /identity/v3/projects/cd7624056a1342a19b08c707ea89a0be => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:24.135540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:24.139824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:24.256269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b61e32b0a4d4dd19b8ab8560e30e43f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4dfe920f9498418bada826fc59afdb91', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:24.257851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.260252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.260748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.261644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:24.263202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:24.266566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:24.286328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 874/1745] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:24.286923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:24.289017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:24.291772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39467e-d2c1-41f3-8ac4-ebcbad26168c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:24.297143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:24.297950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.298295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.298629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] PATH_INFO: `/v3/users/e848173ee2f14b43a1f4fbf95ba600e8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.299936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e848173ee2f14b43a1f4fbf95ba600e8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:24.302871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:24.308699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb8c71-142f-4d2b-976b-97c53d7b7a32 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:24.314319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:24.314954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.315714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.316054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] PATH_INFO: `/v3/projects/8a8b382c6ef34fda8b176a51909be691` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.317285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8b382c6ef34fda8b176a51909be691)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:24.319538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:24.361871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:24.383951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e848173ee2f14b43a1f4fbf95ba600e8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.395999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8b382c6ef34fda8b176a51909be691', 'request_id': 'req-ff5248a7-04b3-404e-910c-e16b19975452'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.406949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8b382c6ef34fda8b176a51909be691', 'request_id': 'req-ff5248a7-04b3-404e-910c-e16b19975452'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.413519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e848173ee2f14b43a1f4fbf95ba600e8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.414282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e848173ee2f14b43a1f4fbf95ba600e8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.416085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8b382c6ef34fda8b176a51909be691', 'request_id': 'req-ff5248a7-04b3-404e-910c-e16b19975452'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.423974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e848173ee2f14b43a1f4fbf95ba600e8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.434298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-fcf1a9dd-e572-4c91-b712-ff2ca1df6b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e848173ee2f14b43a1f4fbf95ba600e8'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.436460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] The token cache is being invalidate because project 8a8b382c6ef34fda8b176a51909be691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:24.436849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.438012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.442812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 875/1746] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:24 2022] DELETE /identity/v3/projects/8a8b382c6ef34fda8b176a51909be691 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:24.442812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:24.442812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:24.452051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 872/1747] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:24 2022] DELETE /identity/v3/users/e848173ee2f14b43a1f4fbf95ba600e8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:24.452051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd9792-7039-4846-8e64-414e369b9403 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.452051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd9792-7039-4846-8e64-414e369b9403 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.452051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd9792-7039-4846-8e64-414e369b9403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.490110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7ffd9792-7039-4846-8e64-414e369b9403 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:24.491541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7ffd9792-7039-4846-8e64-414e369b9403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:24.529036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5248a7-04b3-404e-910c-e16b19975452 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:24.543028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:24.584361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 873/1748] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:24.592503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65fe94-5549-4660-bb2c-288cda42b5a6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.592789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65fe94-5549-4660-bb2c-288cda42b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.593408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65fe94-5549-4660-bb2c-288cda42b5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.618816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8c65fe94-5549-4660-bb2c-288cda42b5a6 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:24.620413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8c65fe94-5549-4660-bb2c-288cda42b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:24.635080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 874/1749] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:24.635855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65fe94-5549-4660-bb2c-288cda42b5a6 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:24.636316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b61e32b0a4d4dd19b8ab8560e30e43f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4dfe920f9498418bada826fc59afdb91', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:24.637260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.637739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.638220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] PATH_INFO: `/v3/projects/2b3bf3acffe44a3fa49292dd54b5056f` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.640250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2b3bf3acffe44a3fa49292dd54b5056f)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:24.643473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:24.648765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65fe94-5549-4660-bb2c-288cda42b5a6 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:24.665634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:24.666689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.667164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.667874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.668690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:24.669056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:24.670910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:24.689947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 875/1750] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:24.691429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:24.707341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603549c-cec1-47ea-b5f5-3aecac8189a7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:24.711991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3bf3acffe44a3fa49292dd54b5056f', 'request_id': 'req-47533514-087b-4fcf-b56c-263f15d46632'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.723304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3bf3acffe44a3fa49292dd54b5056f', 'request_id': 'req-47533514-087b-4fcf-b56c-263f15d46632'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.733216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3bf3acffe44a3fa49292dd54b5056f', 'request_id': 'req-47533514-087b-4fcf-b56c-263f15d46632'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.750232 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] The token cache is being invalidate because project 2b3bf3acffe44a3fa49292dd54b5056f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:24.750661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.751644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:24.754049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 876/1751] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:24 2022] DELETE /identity/v3/projects/2b3bf3acffe44a3fa49292dd54b5056f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:24.792585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:24.793268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.793574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.793843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.794565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:24.796845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:24.837939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 876/1752] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:24 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:24.841736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:24.843295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:24.861050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533514-087b-4fcf-b56c-263f15d46632 tempest-VolumeQuotaClassesTest-664998085 tempest-VolumeQuotaClassesTest-664998085-project-admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:24.949919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:24.950675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:24.951025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:24.951301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:24.951942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:24.953974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:24.988315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 877/1753] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:24 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:24.992463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.011664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71041d60-2005-4074-9403-6b27dc463314 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.025739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.034196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.034653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.035038 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.036512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.043395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.068297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 877/1754] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:25.073012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.085678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0fc9c-dd18-4314-b4a0-9da7aa7ea976 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.093394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.094033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.094349 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.094824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.095469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.097549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.115897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.118720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.129677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:25.141352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 878/1755] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:25.145507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.149270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb459974-8a89-43a8-9f00-b2b8932037d5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.174853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.175528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.175804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.176069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] PATH_INFO: `/v3/users/6ac1c9bbf1bb452fb7cc72c59857ea9e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.179478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a129-0a08-4581-beeb-ff4717b333c3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.185381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac1c9bbf1bb452fb7cc72c59857ea9e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.190793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.193541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.194951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.195442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.197043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] PATH_INFO: `/v3/projects/923e6b02078246a2aea23720bc032e4b/users/ab83307d27a648f9b548b54422a95538/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.239407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=923e6b02078246a2aea23720bc032e4b, user_id=ab83307d27a648f9b548b54422a95538, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.246745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.271183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:25.281249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 879/1756] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:25 2022] PUT /identity/v3/projects/923e6b02078246a2aea23720bc032e4b/users/ab83307d27a648f9b548b54422a95538/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:25.299828 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.302207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.306375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac1c9bbf1bb452fb7cc72c59857ea9e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.327948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac1c9bbf1bb452fb7cc72c59857ea9e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.329199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac1c9bbf1bb452fb7cc72c59857ea9e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.340354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac1c9bbf1bb452fb7cc72c59857ea9e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.355071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cfb3d65-d3dc-4fd1-9019-3b2bd9ed6e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac1c9bbf1bb452fb7cc72c59857ea9e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.374182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 878/1757] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:25 2022] DELETE /identity/v3/users/6ac1c9bbf1bb452fb7cc72c59857ea9e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:25.379649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941457e-7435-4d00-bfcf-53eedc5a7fc3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.379649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941457e-7435-4d00-bfcf-53eedc5a7fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.379649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941457e-7435-4d00-bfcf-53eedc5a7fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.413543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09cb5-87cc-4cdf-907b-56b78780d9ea admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.457827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f941457e-7435-4d00-bfcf-53eedc5a7fc3 None None] MFA Rules not processed for user `ab83307d27a648f9b548b54422a95538`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:25.459041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f941457e-7435-4d00-bfcf-53eedc5a7fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.542062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.542803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.543084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.543354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] PATH_INFO: `/v3/projects/7fce067ad4f54bdfa814eadce2e06111` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.544759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fce067ad4f54bdfa814eadce2e06111)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.547086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.576030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 879/1758] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:25.577513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.578120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.578478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.614135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:25.616593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.627170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 880/1759] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:25.628308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.629836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.645091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26632789-eea8-472e-b39f-94352bbf9d48 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.666384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fce067ad4f54bdfa814eadce2e06111', 'request_id': 'req-7665de76-6598-4a63-8f93-e50affc6f703'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.684726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fce067ad4f54bdfa814eadce2e06111', 'request_id': 'req-7665de76-6598-4a63-8f93-e50affc6f703'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.700960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fce067ad4f54bdfa814eadce2e06111', 'request_id': 'req-7665de76-6598-4a63-8f93-e50affc6f703'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.715886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] The token cache is being invalidate because project 7fce067ad4f54bdfa814eadce2e06111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:25.716183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.718965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:25.724178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 880/1760] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:25 2022] DELETE /identity/v3/projects/7fce067ad4f54bdfa814eadce2e06111 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:25.725089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.730034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.750553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.751518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.751994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.752469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.753301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.757004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.765668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:25.802682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:25.820078 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 881/1761] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:25 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:25.820078 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.821815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.832198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665de76-6598-4a63-8f93-e50affc6f703 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.904301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea82ec8-df63-431d-91de-4aad0eb6e2f8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.913659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.915226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.915506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.915773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] PATH_INFO: `/v3/users/133140bb0b904af4b2cb5330ff6db312` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.917075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=133140bb0b904af4b2cb5330ff6db312)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.917296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:25.918013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:25.918336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:25.918765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:25.919243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.920910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.956271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:25.956271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:25.965734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 881/1762] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:25.966379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:25.968930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:25.986779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921113-b337-46cb-bc38-fe738530155a service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:25.999005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133140bb0b904af4b2cb5330ff6db312'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.001366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.002553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.002837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.003114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.003754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.005751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.016385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133140bb0b904af4b2cb5330ff6db312'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.018132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133140bb0b904af4b2cb5330ff6db312'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.028824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133140bb0b904af4b2cb5330ff6db312'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.048630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 882/1763] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:25 2022] POST /identity/v3/users => generated 293 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:26.048630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.063089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1445f49-f16b-453f-a5e3-bf4987cadc24 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.066900 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133140bb0b904af4b2cb5330ff6db312'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.074049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.075143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.075417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.075838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.076555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.079028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.080638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 882/1764] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:25 2022] DELETE /identity/v3/users/133140bb0b904af4b2cb5330ff6db312 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:26.099583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 883/1765] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:26.100332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.105810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.207194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7af5-92b0-4f58-ba78-b35bb8006a09 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.220989 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.221555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.221832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.222099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] PATH_INFO: `/v3/projects/f6834818371e4f87aac0a5e4d71d4d3c/users/b04072f63a7b42469189b5425fb5c584/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.234940 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:26.249018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6834818371e4f87aac0a5e4d71d4d3c, user_id=b04072f63a7b42469189b5425fb5c584, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.253006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.270823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 883/1766] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:26 2022] PUT /identity/v3/projects/f6834818371e4f87aac0a5e4d71d4d3c/users/b04072f63a7b42469189b5425fb5c584/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:26.275005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.284927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340bb86-f69d-47d6-8309-4bf804f010b0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.371215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.371999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.372126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.372922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.373210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.374814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.392129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 884/1767] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:26.394319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.409125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e857c0d-29c8-42e0-a210-5b3fc144a46f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.416336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c5530-6abb-46c8-8be1-a609e8833378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.417242 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c5530-6abb-46c8-8be1-a609e8833378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.417820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c5530-6abb-46c8-8be1-a609e8833378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.418088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c5530-6abb-46c8-8be1-a609e8833378 admin admin] PATH_INFO: `/v3/projects/f6834818371e4f87aac0a5e4d71d4d3c/users/b04072f63a7b42469189b5425fb5c584/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.421001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c5530-6abb-46c8-8be1-a609e8833378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6834818371e4f87aac0a5e4d71d4d3c, user_id=b04072f63a7b42469189b5425fb5c584, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.423885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c5530-6abb-46c8-8be1-a609e8833378 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.433564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.437121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.448271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 884/1768] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:26 2022] PUT /identity/v3/projects/f6834818371e4f87aac0a5e4d71d4d3c/users/b04072f63a7b42469189b5425fb5c584/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:26.459312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0157bce-80f2-4967-ae5f-cb50893af05a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.459892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.460261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.460698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.497474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] MFA Rules not processed for user `b04072f63a7b42469189b5425fb5c584`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:26.499096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.542877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.543534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.543808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.544122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] PATH_INFO: `/v3/projects/5d1f61d01c3042d78cca1b099e24691c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.545789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1f61d01c3042d78cca1b099e24691c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.548028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.550562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:26.578665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 885/1769] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:26.579516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.581024 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.601726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19445a-09e9-4629-a181-c10304fc82bf None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.603566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1f61d01c3042d78cca1b099e24691c', 'request_id': 'req-6ec19ac3-55a9-4157-8261-2e46de50d96e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.612694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1f61d01c3042d78cca1b099e24691c', 'request_id': 'req-6ec19ac3-55a9-4157-8261-2e46de50d96e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.619530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1f61d01c3042d78cca1b099e24691c', 'request_id': 'req-6ec19ac3-55a9-4157-8261-2e46de50d96e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.631545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] The token cache is being invalidate because project 5d1f61d01c3042d78cca1b099e24691c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:26.631831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.632520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.635057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 885/1770] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:26 2022] DELETE /identity/v3/projects/5d1f61d01c3042d78cca1b099e24691c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:26.635827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.637804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.663319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.663882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.664156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.664485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] PATH_INFO: `/v3/projects/cc94a4d98e554edba293c85ef8474db8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.665589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc94a4d98e554edba293c85ef8474db8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.667467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.697748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec19ac3-55a9-4157-8261-2e46de50d96e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.704516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8928a-a785-4637-94a3-a5542febed7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.705444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8928a-a785-4637-94a3-a5542febed7e service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.705740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8928a-a785-4637-94a3-a5542febed7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.706006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8928a-a785-4637-94a3-a5542febed7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.707864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-92d8928a-a785-4637-94a3-a5542febed7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.721322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc94a4d98e554edba293c85ef8474db8', 'request_id': 'req-1db98385-b4f4-4517-b134-03cfff957c06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.730846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc94a4d98e554edba293c85ef8474db8', 'request_id': 'req-1db98385-b4f4-4517-b134-03cfff957c06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.735303 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:26.737877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc94a4d98e554edba293c85ef8474db8', 'request_id': 'req-1db98385-b4f4-4517-b134-03cfff957c06'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.753422 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] The token cache is being invalidate because project cc94a4d98e554edba293c85ef8474db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:26.753715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.754381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.760444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 886/1771] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:26 2022] DELETE /identity/v3/projects/cc94a4d98e554edba293c85ef8474db8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:26.760444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.760444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.781920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8928a-a785-4637-94a3-a5542febed7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.786249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8928a-a785-4637-94a3-a5542febed7e service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.796208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 886/1772] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:26.801612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb684466-dff6-484c-9786-064b68d5a187 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.801902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb684466-dff6-484c-9786-064b68d5a187 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.802175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb684466-dff6-484c-9786-064b68d5a187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.823951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fb684466-dff6-484c-9786-064b68d5a187 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:26.825180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fb684466-dff6-484c-9786-064b68d5a187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.848262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db98385-b4f4-4517-b134-03cfff957c06 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.857307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.857963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.858663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.858933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] PATH_INFO: `/v3/users/6b61e32b0a4d4dd19b8ab8560e30e43f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.861168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b61e32b0a4d4dd19b8ab8560e30e43f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.863688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.882680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:26.907738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 887/1773] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:26.915873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1af74-8427-49da-9255-8e3fe457ab58 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.916092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1af74-8427-49da-9255-8e3fe457ab58 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.916419 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1af74-8427-49da-9255-8e3fe457ab58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.923839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b61e32b0a4d4dd19b8ab8560e30e43f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.937222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b61e32b0a4d4dd19b8ab8560e30e43f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.937658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-12e1af74-8427-49da-9255-8e3fe457ab58 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:26.937941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b61e32b0a4d4dd19b8ab8560e30e43f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.938851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-12e1af74-8427-49da-9255-8e3fe457ab58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:26.947664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 888/1774] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:26.949981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b61e32b0a4d4dd19b8ab8560e30e43f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.951801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1af74-8427-49da-9255-8e3fe457ab58 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.958762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b61e32b0a4d4dd19b8ab8560e30e43f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:26.961443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1af74-8427-49da-9255-8e3fe457ab58 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:26.966239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:26.966838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:26.966965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 887/1775] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:26 2022] DELETE /identity/v3/users/6b61e32b0a4d4dd19b8ab8560e30e43f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:26.967117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:26.967381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:26.968215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:26.968546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:26.970400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:26.988807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 889/1776] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:25:26.990806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:26.993265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:27.066082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071d3cd-4621-4ee6-afe1-cdfd627c3f5f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.077271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.078486 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.078486 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.078815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.079231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.081174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.098148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:27.125871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.127222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 888/1777] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:26 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:27.141652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d4463-f798-4426-869e-76a4443e5b6f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.246415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.246550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.246804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.247117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.248700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.250832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.290620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.303542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 890/1778] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:27 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:27.306387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c909c50-be7c-4b38-9648-4ee3e23f3874 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.329066 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.332557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.333099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.333561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.335911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.341112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.377648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.391123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4d736-1f6a-4739-892c-67d8478a82db admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.392292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 889/1779] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:27.404185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.404900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.405215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.405518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.406171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.408092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.429679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 891/1780] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:27.434604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.450883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749471ad-2dd5-4551-9bb4-26421702e3e5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.470381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.473321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:27.477341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b532b-4442-428f-b842-0a4e328b5f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.479805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b532b-4442-428f-b842-0a4e328b5f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.480624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b532b-4442-428f-b842-0a4e328b5f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.481118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b532b-4442-428f-b842-0a4e328b5f43 admin admin] PATH_INFO: `/v3/projects/0b9364d7587d4041beadcab24aebb3db/users/5d0168d1b0a34711af06bdbf9ae19a6e/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.522942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb7569-21fc-4325-b0b4-429cbf6ba3b0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.523162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b532b-4442-428f-b842-0a4e328b5f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9364d7587d4041beadcab24aebb3db, user_id=5d0168d1b0a34711af06bdbf9ae19a6e, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.528291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b532b-4442-428f-b842-0a4e328b5f43 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.541940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.542618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.543010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.543294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] PATH_INFO: `/v3/projects/4dfe920f9498418bada826fc59afdb91` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.547363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 890/1781] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:27 2022] PUT /identity/v3/projects/0b9364d7587d4041beadcab24aebb3db/users/5d0168d1b0a34711af06bdbf9ae19a6e/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:27.548541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dfe920f9498418bada826fc59afdb91)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.550882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.560332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.560609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.560880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.603284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] MFA Rules not processed for user `5d0168d1b0a34711af06bdbf9ae19a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:27.604493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:27.631650 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dfe920f9498418bada826fc59afdb91', 'request_id': 'req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:27.640706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dfe920f9498418bada826fc59afdb91', 'request_id': 'req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:27.646967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dfe920f9498418bada826fc59afdb91', 'request_id': 'req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:27.658879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] The token cache is being invalidate because project 4dfe920f9498418bada826fc59afdb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:27.659162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:27.659860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a548c67d-9cfd-4c26-b5e8-b2ae1e13c5c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:27.661465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 892/1782] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:27 2022] DELETE /identity/v3/projects/4dfe920f9498418bada826fc59afdb91 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:27.666787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c083b-b787-4bfe-aa3e-62b3bea61e56 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.667178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c083b-b787-4bfe-aa3e-62b3bea61e56 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.667456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c083b-b787-4bfe-aa3e-62b3bea61e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.682400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 891/1783] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:27.693462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.693641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f00c083b-b787-4bfe-aa3e-62b3bea61e56 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:27.694841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:27.694962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f00c083b-b787-4bfe-aa3e-62b3bea61e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:27.721610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:27.761225 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0da9c-e002-44bf-be9a-e4b75cdc144b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.762655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 893/1784] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:27.771353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb055b9b-239a-4031-9b39-09c959758dfe None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.771648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb055b9b-239a-4031-9b39-09c959758dfe None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.771922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb055b9b-239a-4031-9b39-09c959758dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.796735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eb055b9b-239a-4031-9b39-09c959758dfe None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:27.797970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eb055b9b-239a-4031-9b39-09c959758dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:27.807473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 894/1785] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:27.811462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb055b9b-239a-4031-9b39-09c959758dfe None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.819918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d4ce3e02e5945ffbbde389398b146c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.820571 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.820930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.821207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.823050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb055b9b-239a-4031-9b39-09c959758dfe None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.828109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.828738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.829016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.829282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.829923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:27.830249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.831826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.839834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:27.843877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 895/1786] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:27.848654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.863011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b09ef-77df-4c04-a081-a6f2a76b3990 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:27.871534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:27.881143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:27.881313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:27.881531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:27.882197 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.884086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:27.885042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.886744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:27.896493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 892/1787] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:27.929319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 896/1788] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:27 2022] POST /identity/v3/projects => generated 399 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:27.933495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:27.945116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390995a3-80de-4741-9ffa-8a565825a043 service glance] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.007224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.007925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.009031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.009624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.010320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.012331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.043074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 893/1789] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:27 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:28.045013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.058913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56b81c-6e95-42c0-b51b-59b749c601be admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.063921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.064699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.065042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.065378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.066094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.067818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.080303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 897/1790] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:28.083364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.093859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b43ecb-f461-46d6-92cd-8ff83a587295 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.098639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cf6d4-3de0-4bfc-ba82-13b0dd9aaeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.099226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cf6d4-3de0-4bfc-ba82-13b0dd9aaeba admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.099535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cf6d4-3de0-4bfc-ba82-13b0dd9aaeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.099791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cf6d4-3de0-4bfc-ba82-13b0dd9aaeba admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.100401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cf6d4-3de0-4bfc-ba82-13b0dd9aaeba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.102025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cf6d4-3de0-4bfc-ba82-13b0dd9aaeba admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.112260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 894/1791] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:28.116405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.127667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570890d-0d32-44be-b8ad-43cd102be2b7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.132396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.133034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.133891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.134416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] PATH_INFO: `/v3/projects/bd7342d677a4406d9363116c3b1c035d/users/8a6976604ee9403e8fb313caf484e369/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.149142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7342d677a4406d9363116c3b1c035d, user_id=8a6976604ee9403e8fb313caf484e369, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.151755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.164936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 898/1792] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:28 2022] PUT /identity/v3/projects/bd7342d677a4406d9363116c3b1c035d/users/8a6976604ee9403e8fb313caf484e369/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:28.175169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc654a19-474a-442e-a817-41bc82217317 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.175474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc654a19-474a-442e-a817-41bc82217317 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.175740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc654a19-474a-442e-a817-41bc82217317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.205389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cc654a19-474a-442e-a817-41bc82217317 None None] MFA Rules not processed for user `8a6976604ee9403e8fb313caf484e369`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:28.206513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cc654a19-474a-442e-a817-41bc82217317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:28.289756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 895/1793] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:28.297146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.392064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857ff41-6ed6-40f5-88ac-28f9e10f46df admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.411809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.412568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.413046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.413569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.414218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.416968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.452284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 899/1794] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:28 2022] POST /identity/v3/projects => generated 401 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:28.454956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc654a19-474a-442e-a817-41bc82217317 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.465933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc654a19-474a-442e-a817-41bc82217317 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.497873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:28.531279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.531887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.532165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.532462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.533109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.534999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.556445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 896/1795] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:28 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:28.560445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.570713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bf089-4f91-4414-9ea3-95c52e550a05 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.575325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.575930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.576206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.576518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.577325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.578926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.609887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 900/1796] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:28.616673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.631524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc065710-4142-43a5-837f-79ac016da2fd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.637539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.638266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.638543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.638859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] PATH_INFO: `/v3/projects/52d3e276b51f44ec999dc01dd9ec11bd/users/d650e6c199754ebe9305023979d0b4ca/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.658757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d3e276b51f44ec999dc01dd9ec11bd, user_id=d650e6c199754ebe9305023979d0b4ca, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.666214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.690597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 897/1797] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:28 2022] PUT /identity/v3/projects/52d3e276b51f44ec999dc01dd9ec11bd/users/d650e6c199754ebe9305023979d0b4ca/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:28.694751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.709426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b2af-c557-4577-84f7-3c7f77119d20 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.746485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:28.796984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382faeff-f82d-457b-b5f0-7181fb8b4a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.797605 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382faeff-f82d-457b-b5f0-7181fb8b4a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.797881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382faeff-f82d-457b-b5f0-7181fb8b4a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.798152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382faeff-f82d-457b-b5f0-7181fb8b4a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.798853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382faeff-f82d-457b-b5f0-7181fb8b4a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.800542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382faeff-f82d-457b-b5f0-7181fb8b4a43 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.816265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 901/1798] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:28.818260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.828889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfbb18-a9b2-475f-9394-c0cbd7649e5f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:28.833533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:28.834176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.834447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.834710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] PATH_INFO: `/v3/projects/52d3e276b51f44ec999dc01dd9ec11bd/users/d650e6c199754ebe9305023979d0b4ca/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.837227 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d3e276b51f44ec999dc01dd9ec11bd, user_id=d650e6c199754ebe9305023979d0b4ca, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:28.839848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:28.850372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 898/1799] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:28 2022] PUT /identity/v3/projects/52d3e276b51f44ec999dc01dd9ec11bd/users/d650e6c199754ebe9305023979d0b4ca/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:28.862206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6f81-9dea-40fe-b7ef-520cba7cd563 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:28.862490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6f81-9dea-40fe-b7ef-520cba7cd563 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:28.862761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6f81-9dea-40fe-b7ef-520cba7cd563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:28.890859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ee7e6f81-9dea-40fe-b7ef-520cba7cd563 None None] MFA Rules not processed for user `d650e6c199754ebe9305023979d0b4ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:28.892041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ee7e6f81-9dea-40fe-b7ef-520cba7cd563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:28.960478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 902/1800] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:28.971039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:28.972717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:29.033034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18003e8c-7872-48b3-8707-99facd8ece4a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:29.036590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:29.037243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:29.037533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:29.037785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:29.047582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:29.049242 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:29.058977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 899/1801] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:29.142884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e6f81-9dea-40fe-b7ef-520cba7cd563 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:29.152848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e6f81-9dea-40fe-b7ef-520cba7cd563 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:29.157860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21abac9-cd5d-4120-aad9-daf76fba96b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:29.158947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21abac9-cd5d-4120-aad9-daf76fba96b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:29.158947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21abac9-cd5d-4120-aad9-daf76fba96b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:29.158947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21abac9-cd5d-4120-aad9-daf76fba96b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:29.224772 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21abac9-cd5d-4120-aad9-daf76fba96b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:29.226434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21abac9-cd5d-4120-aad9-daf76fba96b5 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:29.237164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 903/1802] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:29.631385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:29.633876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:29.768463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651afcf-1b96-48e1-8e4d-4735932864ee service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:29.774991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:29.776076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:29.776076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:29.776320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] PATH_INFO: `/v3/users/ab83307d27a648f9b548b54422a95538` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:29.777867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab83307d27a648f9b548b54422a95538)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:29.780030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:29.810300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:29.859398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab83307d27a648f9b548b54422a95538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:29.893463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab83307d27a648f9b548b54422a95538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:29.894945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab83307d27a648f9b548b54422a95538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:29.908521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab83307d27a648f9b548b54422a95538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:29.933525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab83307d27a648f9b548b54422a95538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:29.943779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 900/1803] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:29 2022] DELETE /identity/v3/users/ab83307d27a648f9b548b54422a95538 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:29.948485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:29.948949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:29.949125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:29.980843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:29.986536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:30.059955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:30.083627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 904/1804] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:30.093523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:30.094936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:30.156846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd235d9-8325-4866-b99b-c352158e32f8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:30.160721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:30.161334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:30.161608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:30.161887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:30.174202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:30.175631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:30.185700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 901/1805] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:30.513278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:30.514807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:30.536475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11857-97ce-4707-ae31-7e341a8e34de None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:30.554208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:30.555424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:30.555773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:30.556182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] PATH_INFO: `/v3/projects/923e6b02078246a2aea23720bc032e4b` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:30.557661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=923e6b02078246a2aea23720bc032e4b)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:30.559796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:30.625322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923e6b02078246a2aea23720bc032e4b', 'request_id': 'req-2e7cb62e-127e-40c9-82fb-38609cbd2b39'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.634783 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923e6b02078246a2aea23720bc032e4b', 'request_id': 'req-2e7cb62e-127e-40c9-82fb-38609cbd2b39'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.641945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923e6b02078246a2aea23720bc032e4b', 'request_id': 'req-2e7cb62e-127e-40c9-82fb-38609cbd2b39'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.655594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] The token cache is being invalidate because project 923e6b02078246a2aea23720bc032e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:30.655900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.656865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.658563 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 905/1806] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:30 2022] DELETE /identity/v3/projects/923e6b02078246a2aea23720bc032e4b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:30.662412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:30.671921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f36cda-357f-4f57-92ad-7f667ea0eb8f service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:30.748384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:30.749034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:30.749310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:30.749587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] PATH_INFO: `/v3/users/b04072f63a7b42469189b5425fb5c584` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:30.750740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04072f63a7b42469189b5425fb5c584)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:30.752947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:30.813323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:30.815213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04072f63a7b42469189b5425fb5c584'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.831475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04072f63a7b42469189b5425fb5c584'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.833003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04072f63a7b42469189b5425fb5c584'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.841959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04072f63a7b42469189b5425fb5c584'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.850831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-643cf8b5-8829-4aa9-a152-f467b906cb1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04072f63a7b42469189b5425fb5c584'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:30.860720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 902/1807] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:30 2022] DELETE /identity/v3/users/b04072f63a7b42469189b5425fb5c584 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:31.147783 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:31.149903 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:31.175184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cb62e-127e-40c9-82fb-38609cbd2b39 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:31.293098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:31.294503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:31.294773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:31.295396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] PATH_INFO: `/v3/users/2cbfebb0012d45478daf865dbb3b8209` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:31.298227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cbfebb0012d45478daf865dbb3b8209)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:31.300862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:31.395367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cbfebb0012d45478daf865dbb3b8209'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.416333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cbfebb0012d45478daf865dbb3b8209'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.417254 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cbfebb0012d45478daf865dbb3b8209'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.431042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:31.433483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cbfebb0012d45478daf865dbb3b8209'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.451138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cbfebb0012d45478daf865dbb3b8209'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.458454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 906/1808] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:31 2022] DELETE /identity/v3/users/2cbfebb0012d45478daf865dbb3b8209 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:31.463732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:31.464126 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:31.464542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:31.486817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:31.488218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:31.583998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 903/1809] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:31.619348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:31.620756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:31.698328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:31.700074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:31.723872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a778-73b0-4321-bee4-40745525fc95 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:31.728931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f0192-a2fc-4cf3-88db-47d33ea3c488 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:31.729773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:31.730708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:31.731058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:31.731512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] PATH_INFO: `/v3/projects/f6834818371e4f87aac0a5e4d71d4d3c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:31.733121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6834818371e4f87aac0a5e4d71d4d3c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:31.735110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:31.736182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff4dd7-d4cb-41b3-b441-d0a7fdeb36de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:31.740506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff4dd7-d4cb-41b3-b441-d0a7fdeb36de service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:31.741054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff4dd7-d4cb-41b3-b441-d0a7fdeb36de service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:31.741540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff4dd7-d4cb-41b3-b441-d0a7fdeb36de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:31.777198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff4dd7-d4cb-41b3-b441-d0a7fdeb36de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:31.783870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff4dd7-d4cb-41b3-b441-d0a7fdeb36de service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:31.797617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 907/1810] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:31.817142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6834818371e4f87aac0a5e4d71d4d3c', 'request_id': 'req-01537b05-635a-49b8-8698-517ef2926696'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.829433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6834818371e4f87aac0a5e4d71d4d3c', 'request_id': 'req-01537b05-635a-49b8-8698-517ef2926696'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.842719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6834818371e4f87aac0a5e4d71d4d3c', 'request_id': 'req-01537b05-635a-49b8-8698-517ef2926696'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.856599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] The token cache is being invalidate because project f6834818371e4f87aac0a5e4d71d4d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:31.857012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.857592 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01537b05-635a-49b8-8698-517ef2926696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:31.859239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 904/1811] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:31 2022] DELETE /identity/v3/projects/f6834818371e4f87aac0a5e4d71d4d3c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:31.864492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd700c3f-c642-47a8-ae79-d60a62bb1a30 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:31.864996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd700c3f-c642-47a8-ae79-d60a62bb1a30 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:31.865446 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd700c3f-c642-47a8-ae79-d60a62bb1a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:31.887725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fd700c3f-c642-47a8-ae79-d60a62bb1a30 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:31.889005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fd700c3f-c642-47a8-ae79-d60a62bb1a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:31.954643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 908/1812] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:31.962542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc072b-81e7-4f2c-8696-1ba5f977d88d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:31.962885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc072b-81e7-4f2c-8696-1ba5f977d88d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:31.963163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc072b-81e7-4f2c-8696-1ba5f977d88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:31.982538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3efc072b-81e7-4f2c-8696-1ba5f977d88d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:31.983721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3efc072b-81e7-4f2c-8696-1ba5f977d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:31.991649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 905/1813] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:31.996780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd700c3f-c642-47a8-ae79-d60a62bb1a30 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.008643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd700c3f-c642-47a8-ae79-d60a62bb1a30 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.014064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.014720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.014999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.015321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.015924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:32.016244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.017844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.030484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 909/1814] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:32.035136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc072b-81e7-4f2c-8696-1ba5f977d88d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.044395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc072b-81e7-4f2c-8696-1ba5f977d88d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.050942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.051543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.051814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.052076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.052738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.059914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.096204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 906/1815] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:32.100209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.110126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408600d-8bac-4711-9fee-c0537efe094a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.156412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.159619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:32.182692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:32.182829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.183423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.183831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.184176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.185128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.187502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.213226 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 910/1816] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:32.217185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.226407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e007b-412a-4e20-abd4-00674a1c132e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.231549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.232129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.232439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.232705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.233321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.234622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae232954-17d7-49d0-b83c-f13c36fa16e5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.234873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.237015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:32.239518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.240152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.240466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.240736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] PATH_INFO: `/v3/projects/aea0cf9918594b1bbaee3a89da127eff` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.241863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea0cf9918594b1bbaee3a89da127eff)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.244352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.251465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 911/1817] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:32.252465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.263487 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a2dce-d004-447f-acf4-95d72e2dac56 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.268456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.269056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.269337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.269607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.270586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.271803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.282717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 912/1818] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:32.286532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.298501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b49934-2d58-45bc-aec7-6c5b0d6298b5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.302744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea0cf9918594b1bbaee3a89da127eff', 'request_id': 'req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:32.314877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea0cf9918594b1bbaee3a89da127eff', 'request_id': 'req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:32.321011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea0cf9918594b1bbaee3a89da127eff', 'request_id': 'req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:32.335668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] The token cache is being invalidate because project aea0cf9918594b1bbaee3a89da127eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:32.336633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:32.337478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-6bd6f91e-d78c-47f9-b7e1-525f13518d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:32.339356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 907/1819] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:32 2022] DELETE /identity/v3/projects/aea0cf9918594b1bbaee3a89da127eff => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:32.344331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa73d09-b6e8-461c-a634-662ed46c56c0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.344616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa73d09-b6e8-461c-a634-662ed46c56c0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.344887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa73d09-b6e8-461c-a634-662ed46c56c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.373329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-afa73d09-b6e8-461c-a634-662ed46c56c0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:32.374493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c3716-ea63-4496-a0a9-11293ac002f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.374643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-afa73d09-b6e8-461c-a634-662ed46c56c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:32.375164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3716-ea63-4496-a0a9-11293ac002f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.375526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3716-ea63-4496-a0a9-11293ac002f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.375968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3716-ea63-4496-a0a9-11293ac002f7 admin admin] PATH_INFO: `/v3/projects/ad0ac3d8983248bcad9b84c1be67bc47/users/e89e34bc3817448d93d576348bd346de/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.388395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 908/1820] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:32.396296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638323-8eda-4060-9102-55f5a4883c87 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.396668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638323-8eda-4060-9102-55f5a4883c87 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.397003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7638323-8eda-4060-9102-55f5a4883c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.402341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c3716-ea63-4496-a0a9-11293ac002f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0ac3d8983248bcad9b84c1be67bc47, user_id=e89e34bc3817448d93d576348bd346de, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.405545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c3716-ea63-4496-a0a9-11293ac002f7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.417876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b7638323-8eda-4060-9102-55f5a4883c87 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:32.419189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b7638323-8eda-4060-9102-55f5a4883c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:32.422638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 913/1821] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:32 2022] PUT /identity/v3/projects/ad0ac3d8983248bcad9b84c1be67bc47/users/e89e34bc3817448d93d576348bd346de/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:32.435862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.436151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.436615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.474220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] MFA Rules not processed for user `e89e34bc3817448d93d576348bd346de`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:32.475482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:32.490059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 909/1822] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:32.497364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7638323-8eda-4060-9102-55f5a4883c87 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.507057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7638323-8eda-4060-9102-55f5a4883c87 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.512360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.513064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.513348 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.513622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.514259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:32.514656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.516270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.530393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 910/1823] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:32.534776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.545687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f6936-8523-4130-9daf-90e7112ff604 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.550698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.551560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.551840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.552112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.552788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.554774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.564711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 914/1824] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:32.584669 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.585821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:32.604278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 911/1825] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/projects => generated 397 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:32.608208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.621011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:32.649464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547a539-0afd-4a5e-b97d-f1f6a5df90d8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.663692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:32.680479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ad939-9925-445c-9ecf-23f68b48f67f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.685412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.686485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.686771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.687204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.687973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.689904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.706426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cbd13-7988-4fec-b178-a9894fae726a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.707044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cbd13-7988-4fec-b178-a9894fae726a service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.707317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cbd13-7988-4fec-b178-a9894fae726a service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.707592 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cbd13-7988-4fec-b178-a9894fae726a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.723921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 912/1826] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:32.728130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.737969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80395-3587-4381-9138-343275e5bf0b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.743260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.743859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.744137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.744440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.745076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.746657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.759836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 913/1827] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:32.762406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.773563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9c0a-8b17-4a9e-8e98-6250ecbaac1a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.780919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.781564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.781842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.782118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.782763 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.785410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cbd13-7988-4fec-b178-a9894fae726a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.785561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.787453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cbd13-7988-4fec-b178-a9894fae726a service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.798343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 914/1828] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:32.803475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:32.809400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 915/1829] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:32.815081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beed7b-2311-4ed7-9791-b60a4cf76474 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:32.821863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:32.822533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.822807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.823077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] PATH_INFO: `/v3/projects/cc29c7be9aff419f97301a95eab7755e/users/85d4be5477d54a2bba6cfb0a7e59c4b5/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.846816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc29c7be9aff419f97301a95eab7755e, user_id=85d4be5477d54a2bba6cfb0a7e59c4b5, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:32.850534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:32.883726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 915/1830] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:32 2022] PUT /identity/v3/projects/cc29c7be9aff419f97301a95eab7755e/users/85d4be5477d54a2bba6cfb0a7e59c4b5/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:32.885753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:32.886994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:32.887320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:32.947136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] MFA Rules not processed for user `85d4be5477d54a2bba6cfb0a7e59c4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:32.949093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:33.055315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 916/1831] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:33.076557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:33.078785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:33.241337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb193b8-9942-473b-8520-8a41fa7f47ef admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:33.245173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cb627-4d45-45f3-8b18-1d8a346600e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:33.245714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb627-4d45-45f3-8b18-1d8a346600e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:33.245997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb627-4d45-45f3-8b18-1d8a346600e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:33.246270 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb627-4d45-45f3-8b18-1d8a346600e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:33.262981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cb627-4d45-45f3-8b18-1d8a346600e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:33.264722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cb627-4d45-45f3-8b18-1d8a346600e6 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:33.279420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 916/1832] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:33.791237 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:33.792709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:33.862138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a5a80-327c-47f9-bd92-fd680070c5d0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:33.867910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:33.868693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:33.868929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:33.869246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] PATH_INFO: `/v3/users/8a6976604ee9403e8fb313caf484e369` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:33.870359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6976604ee9403e8fb313caf484e369)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:33.872320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:33.873927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:33.928823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6976604ee9403e8fb313caf484e369'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:33.941514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6976604ee9403e8fb313caf484e369'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:33.942770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6976604ee9403e8fb313caf484e369'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:33.950428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6976604ee9403e8fb313caf484e369'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:33.958450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6976604ee9403e8fb313caf484e369'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:33.964700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 917/1833] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:33 2022] DELETE /identity/v3/users/8a6976604ee9403e8fb313caf484e369 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:33.969410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93220b6-ca29-4b93-84ab-a42791e5d4b2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:33.969937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93220b6-ca29-4b93-84ab-a42791e5d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:33.970136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93220b6-ca29-4b93-84ab-a42791e5d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:33.989382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c93220b6-ca29-4b93-84ab-a42791e5d4b2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:33.990569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c93220b6-ca29-4b93-84ab-a42791e5d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:34.039929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:34.052619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 917/1834] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:34.062814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:34.064212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:34.122259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165f553-258d-45e2-91c2-0746268aa032 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:34.126820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:34.127806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:34.128155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:34.128516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:34.140572 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:34.141993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:34.156084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 918/1835] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:34.308015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db336a2-19e3-406c-95be-b4e47998e0e3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:34.308455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db336a2-19e3-406c-95be-b4e47998e0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:34.308846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db336a2-19e3-406c-95be-b4e47998e0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:34.330015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3db336a2-19e3-406c-95be-b4e47998e0e3 None None] MFA Rules not processed for user `e89e34bc3817448d93d576348bd346de`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:34.331396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3db336a2-19e3-406c-95be-b4e47998e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:34.399692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 918/1836] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:34.669203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:34.670657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:34.683111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7f6a5-175d-4f5b-b196-32d458d2885d service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:34.687711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:34.688348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:34.688626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:34.688892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] PATH_INFO: `/v3/projects/bd7342d677a4406d9363116c3b1c035d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:34.689993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7342d677a4406d9363116c3b1c035d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:34.691866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:34.745792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7342d677a4406d9363116c3b1c035d', 'request_id': 'req-118f451d-115b-4f32-9e8d-708d7e08dc0e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.753548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7342d677a4406d9363116c3b1c035d', 'request_id': 'req-118f451d-115b-4f32-9e8d-708d7e08dc0e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.760308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7342d677a4406d9363116c3b1c035d', 'request_id': 'req-118f451d-115b-4f32-9e8d-708d7e08dc0e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.771609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] The token cache is being invalidate because project bd7342d677a4406d9363116c3b1c035d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:34.771891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.772568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.774769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 919/1837] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:34 2022] DELETE /identity/v3/projects/bd7342d677a4406d9363116c3b1c035d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:34.778887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db336a2-19e3-406c-95be-b4e47998e0e3 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:34.792693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db336a2-19e3-406c-95be-b4e47998e0e3 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:34.858293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:34.858293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:34.858293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:34.858783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] PATH_INFO: `/v3/users/d650e6c199754ebe9305023979d0b4ca` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:34.859573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d650e6c199754ebe9305023979d0b4ca)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:34.861634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:34.940445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd650e6c199754ebe9305023979d0b4ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.959731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd650e6c199754ebe9305023979d0b4ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.960840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd650e6c199754ebe9305023979d0b4ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.982880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd650e6c199754ebe9305023979d0b4ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:34.994657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5fa3b4ec-7da2-44c3-8d51-38242551b9a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd650e6c199754ebe9305023979d0b4ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:35.003745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 919/1838] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:34 2022] DELETE /identity/v3/users/d650e6c199754ebe9305023979d0b4ca => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:35.334722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:35.336484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:35.352546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:35.359960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f451d-115b-4f32-9e8d-708d7e08dc0e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:35.434997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:35.436928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.436928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.436928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] PATH_INFO: `/v3/projects/52d3e276b51f44ec999dc01dd9ec11bd` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.438195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d3e276b51f44ec999dc01dd9ec11bd)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:35.440720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:35.512398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d3e276b51f44ec999dc01dd9ec11bd', 'request_id': 'req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:35.526821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d3e276b51f44ec999dc01dd9ec11bd', 'request_id': 'req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:35.536967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d3e276b51f44ec999dc01dd9ec11bd', 'request_id': 'req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:35.555443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] The token cache is being invalidate because project 52d3e276b51f44ec999dc01dd9ec11bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:35.555735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:35.556418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f3d34f77-b9b8-4906-bb8d-cdf5cd3ef2ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:35.558548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 920/1839] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:35 2022] DELETE /identity/v3/projects/52d3e276b51f44ec999dc01dd9ec11bd => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:35.563461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eeb3f-e3b6-4483-8e60-0049b4c2f2e0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.563741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eeb3f-e3b6-4483-8e60-0049b4c2f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.564181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eeb3f-e3b6-4483-8e60-0049b4c2f2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.587530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a91eeb3f-e3b6-4483-8e60-0049b4c2f2e0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:35.588830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a91eeb3f-e3b6-4483-8e60-0049b4c2f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:35.623131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:35.686616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 920/1840] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:35.693418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874749f-e94c-45ee-b084-5f4b4cf9f05c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.693777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874749f-e94c-45ee-b084-5f4b4cf9f05c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.694119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874749f-e94c-45ee-b084-5f4b4cf9f05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.714288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3874749f-e94c-45ee-b084-5f4b4cf9f05c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:35.715491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3874749f-e94c-45ee-b084-5f4b4cf9f05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:35.723341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 921/1841] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:35.727567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eeb3f-e3b6-4483-8e60-0049b4c2f2e0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:35.738095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eeb3f-e3b6-4483-8e60-0049b4c2f2e0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:35.743089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:35.743697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.743967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.744257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.744929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:35.745223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:35.746795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:35.759522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 921/1842] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:35.763714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874749f-e94c-45ee-b084-5f4b4cf9f05c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:35.772797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874749f-e94c-45ee-b084-5f4b4cf9f05c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:35.777436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:35.778025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.778379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.778657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.779328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:35.781336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:35.811414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 922/1843] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:35 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:35.815362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:35.825800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4108-bd2e-4575-b610-9bc0a5773e93 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:35.883492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:35.884392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.884669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.884938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.885573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:35.887490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:35.912298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 922/1844] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:35 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:35.913835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:35.926009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf87f9-c469-4ce0-b915-fbd8d15f9df2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:35.931606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:35.932279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.932578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.932848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.933485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:35.935066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:35.945455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 923/1845] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:35.949494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:35.959826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9efc0-3ce5-4ffc-8fdb-1c9fe17cabf5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:35.964800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25603600-d27a-4bf2-b345-495c4d4a3b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:35.965393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25603600-d27a-4bf2-b345-495c4d4a3b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:35.965672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25603600-d27a-4bf2-b345-495c4d4a3b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:35.965940 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25603600-d27a-4bf2-b345-495c4d4a3b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:35.966566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25603600-d27a-4bf2-b345-495c4d4a3b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:35.968162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25603600-d27a-4bf2-b345-495c4d4a3b7b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:35.979406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 923/1846] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:35.983954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:35.995196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd86bad-2374-4d41-8b41-c21cea480878 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:35.999945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.000662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.000940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.001213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] PATH_INFO: `/v3/projects/64c16f3e191c4d3ead6ccf5a6304e331/users/295d9d03e0f84d608cbb7d17dc0b6b93/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.016477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c16f3e191c4d3ead6ccf5a6304e331, user_id=295d9d03e0f84d608cbb7d17dc0b6b93, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.019139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.032578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 924/1847] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:35 2022] PUT /identity/v3/projects/64c16f3e191c4d3ead6ccf5a6304e331/users/295d9d03e0f84d608cbb7d17dc0b6b93/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:36.043623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46b056-1a17-4d07-b6d9-a9e19f141061 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.044139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46b056-1a17-4d07-b6d9-a9e19f141061 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.044139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46b056-1a17-4d07-b6d9-a9e19f141061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.072042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bd46b056-1a17-4d07-b6d9-a9e19f141061 None None] MFA Rules not processed for user `295d9d03e0f84d608cbb7d17dc0b6b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:36.073267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bd46b056-1a17-4d07-b6d9-a9e19f141061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:36.139204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 924/1848] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:36.143681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:36.177770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:36.209239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910bf4a-3914-4b29-8401-7bb529e215b9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:36.213921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.214525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.214804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.215074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.215741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.217654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.244623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 925/1849] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:36 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:36.248715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46b056-1a17-4d07-b6d9-a9e19f141061 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:36.259371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46b056-1a17-4d07-b6d9-a9e19f141061 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:36.268874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:36.316823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.317431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.317711 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.317984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.318614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.320599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.343795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 925/1850] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:36 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:36.347956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:36.358748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999362ff-88cd-4e15-9fb0-96de08e8e900 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:36.363354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.364025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.364350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.364679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.365346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.366958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.378792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 926/1851] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:36.383351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:36.393768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af5b55-805b-4681-9d34-136289f45874 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:36.398609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.399256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.399651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.399925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] PATH_INFO: `/v3/projects/88bb33b911d740a8a1e031748aba9873/users/16d89b5dd93f485599f424c72f75efe6/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.415924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88bb33b911d740a8a1e031748aba9873, user_id=16d89b5dd93f485599f424c72f75efe6, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.418587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.431968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 926/1852] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:36 2022] PUT /identity/v3/projects/88bb33b911d740a8a1e031748aba9873/users/16d89b5dd93f485599f424c72f75efe6/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:36.436027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:36.446531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ad017-18ae-4f0e-b584-a68249066191 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:36.504958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0458e9-b2e2-4e68-a2c9-5cd1cf803c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.505477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0458e9-b2e2-4e68-a2c9-5cd1cf803c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.505753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0458e9-b2e2-4e68-a2c9-5cd1cf803c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.506030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0458e9-b2e2-4e68-a2c9-5cd1cf803c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.506661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0458e9-b2e2-4e68-a2c9-5cd1cf803c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.508875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0458e9-b2e2-4e68-a2c9-5cd1cf803c01 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.518831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 927/1853] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:36.523014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:36.534620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db1bc3-39da-4218-97eb-966600fae694 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:36.539622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.540432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.540712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.540989 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] PATH_INFO: `/v3/projects/88bb33b911d740a8a1e031748aba9873/users/16d89b5dd93f485599f424c72f75efe6/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.542890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=88bb33b911d740a8a1e031748aba9873, user_id=16d89b5dd93f485599f424c72f75efe6, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.545630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.556749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 927/1854] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:36 2022] PUT /identity/v3/projects/88bb33b911d740a8a1e031748aba9873/users/16d89b5dd93f485599f424c72f75efe6/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:36.567654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3ff09-be06-4b49-a11e-9139092ea999 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.567932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3ff09-be06-4b49-a11e-9139092ea999 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.568202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3ff09-be06-4b49-a11e-9139092ea999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.602777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9de3ff09-be06-4b49-a11e-9139092ea999 None None] MFA Rules not processed for user `16d89b5dd93f485599f424c72f75efe6`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:36.603975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9de3ff09-be06-4b49-a11e-9139092ea999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:36.678866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 928/1855] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:36.690558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:36.692650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:36.763852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b57fcb-f902-4783-af0e-9fc20bbbdebc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:36.768127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935b4c7-744d-490b-a48b-54d3c3d78ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:36.768781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935b4c7-744d-490b-a48b-54d3c3d78ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:36.769060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935b4c7-744d-490b-a48b-54d3c3d78ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:36.769345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935b4c7-744d-490b-a48b-54d3c3d78ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:36.778634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935b4c7-744d-490b-a48b-54d3c3d78ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:36.780056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935b4c7-744d-490b-a48b-54d3c3d78ca3 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:36.790264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 928/1856] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:37.403344 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3ff09-be06-4b49-a11e-9139092ea999 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:37.434398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:37.467809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3ff09-be06-4b49-a11e-9139092ea999 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:37.472320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:37.472943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:37.473219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:37.473488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] PATH_INFO: `/v3/users/295d9d03e0f84d608cbb7d17dc0b6b93` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:37.474590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=295d9d03e0f84d608cbb7d17dc0b6b93)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:37.477080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:37.523609 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '295d9d03e0f84d608cbb7d17dc0b6b93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:37.535883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '295d9d03e0f84d608cbb7d17dc0b6b93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:37.536604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '295d9d03e0f84d608cbb7d17dc0b6b93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:37.543854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '295d9d03e0f84d608cbb7d17dc0b6b93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:37.551183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '295d9d03e0f84d608cbb7d17dc0b6b93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:37.558600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 929/1857] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:37 2022] DELETE /identity/v3/users/295d9d03e0f84d608cbb7d17dc0b6b93 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:37.563174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:37.563618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:37.563707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:37.582034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:37.583200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:37.643998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 929/1858] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:37.653753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:37.655293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:37.715032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff2f4-a0a6-4745-bb7a-09128875d0cf admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:37.718897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:37.719556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:37.719837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:37.720105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:37.731210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:37.732697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:37.743096 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 930/1859] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:38.155766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:38.157308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:38.169597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa67c68-6537-4337-9c56-4d47e2d697f2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:38.174239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:38.174951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:38.175296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:38.175678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] PATH_INFO: `/v3/projects/64c16f3e191c4d3ead6ccf5a6304e331` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:38.176908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c16f3e191c4d3ead6ccf5a6304e331)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:38.178866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:38.182182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:38.237256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c16f3e191c4d3ead6ccf5a6304e331', 'request_id': 'req-e3b7bb0b-9702-4690-940f-99430633bfab'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.246207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c16f3e191c4d3ead6ccf5a6304e331', 'request_id': 'req-e3b7bb0b-9702-4690-940f-99430633bfab'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.252219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c16f3e191c4d3ead6ccf5a6304e331', 'request_id': 'req-e3b7bb0b-9702-4690-940f-99430633bfab'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.268425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] The token cache is being invalidate because project 64c16f3e191c4d3ead6ccf5a6304e331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:38.268804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.269512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.271087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 930/1860] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:38 2022] DELETE /identity/v3/projects/64c16f3e191c4d3ead6ccf5a6304e331 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:38.274998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:38.284675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670f7ca-46ae-40ba-b23b-7a38e82a0ff4 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:38.340216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:38.355504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:38.356256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:38.356590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:38.356962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] PATH_INFO: `/v3/users/16d89b5dd93f485599f424c72f75efe6` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:38.358089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d89b5dd93f485599f424c72f75efe6)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:38.359999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:38.416759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d89b5dd93f485599f424c72f75efe6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.428973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d89b5dd93f485599f424c72f75efe6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.429689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d89b5dd93f485599f424c72f75efe6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.436571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d89b5dd93f485599f424c72f75efe6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.444597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-5d6bee5e-0ff1-4fd5-98a8-b2bc301e654a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d89b5dd93f485599f424c72f75efe6'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.451499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 931/1861] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:38 2022] DELETE /identity/v3/users/16d89b5dd93f485599f424c72f75efe6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:38.769793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:38.775438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:38.788550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7bb0b-9702-4690-940f-99430633bfab admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:38.846992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:38.847635 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:38.847911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:38.848180 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] PATH_INFO: `/v3/projects/88bb33b911d740a8a1e031748aba9873` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:38.849329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88bb33b911d740a8a1e031748aba9873)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:38.851856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:38.897374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88bb33b911d740a8a1e031748aba9873', 'request_id': 'req-bc225b8f-720d-4850-b302-33a8415a8d59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.905043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88bb33b911d740a8a1e031748aba9873', 'request_id': 'req-bc225b8f-720d-4850-b302-33a8415a8d59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.910855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88bb33b911d740a8a1e031748aba9873', 'request_id': 'req-bc225b8f-720d-4850-b302-33a8415a8d59'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.922158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] The token cache is being invalidate because project 88bb33b911d740a8a1e031748aba9873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:38.922616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.923407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bc225b8f-720d-4850-b302-33a8415a8d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:38.929660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 931/1862] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:38 2022] DELETE /identity/v3/projects/88bb33b911d740a8a1e031748aba9873 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:38.931312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248b9e5-6f64-4bb4-861a-aa8b541bee81 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:38.931615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248b9e5-6f64-4bb4-861a-aa8b541bee81 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:38.931866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248b9e5-6f64-4bb4-861a-aa8b541bee81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:38.952007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f248b9e5-6f64-4bb4-861a-aa8b541bee81 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:38.953245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f248b9e5-6f64-4bb4-861a-aa8b541bee81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:39.026233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 932/1863] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:39.034534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82474e-44d0-495a-bcbc-69c707707c30 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.034824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82474e-44d0-495a-bcbc-69c707707c30 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.035096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82474e-44d0-495a-bcbc-69c707707c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.060283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-fc82474e-44d0-495a-bcbc-69c707707c30 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:39.060283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-fc82474e-44d0-495a-bcbc-69c707707c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:39.072137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 932/1864] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:39.076181 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248b9e5-6f64-4bb4-861a-aa8b541bee81 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.087179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248b9e5-6f64-4bb4-861a-aa8b541bee81 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.092879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.093513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.093911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.094185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.094835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:39.095127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.096749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.116516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 933/1865] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:25:39.118891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82474e-44d0-495a-bcbc-69c707707c30 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.128611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82474e-44d0-495a-bcbc-69c707707c30 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.133670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.134534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.134804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.135093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.135715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.137733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.153364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:39.175870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 933/1866] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:39 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:39.180429 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.211568 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354341c-6485-40a6-a43b-ca6a61a68927 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.259635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:39.267605 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.269853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.271119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.271119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.272119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.274791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.309573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 934/1867] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:39 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:39.313849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.324389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbed2a-c777-47d4-a0a3-c7b8976574ab admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.329383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.330188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.330469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.330800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.331435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.333097 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.344523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 934/1868] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:39.348505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.361159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ff94a-7a6f-4015-8df8-b2274499402a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.368142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae6a1b-c7ed-473b-a95c-bfaaaeef5b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.368952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae6a1b-c7ed-473b-a95c-bfaaaeef5b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.369296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae6a1b-c7ed-473b-a95c-bfaaaeef5b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.369637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae6a1b-c7ed-473b-a95c-bfaaaeef5b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.370385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae6a1b-c7ed-473b-a95c-bfaaaeef5b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.372054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae6a1b-c7ed-473b-a95c-bfaaaeef5b2a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.391545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 935/1869] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:39.394528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.405525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd01e45-396b-42d1-a7bd-e3008d473953 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.410362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.411018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.411300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.411568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] PATH_INFO: `/v3/projects/a7fe20a198504ec49b892baf0c7dbfbf/users/28c7571632c94d23828dde654e2f54f4/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.432488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7fe20a198504ec49b892baf0c7dbfbf, user_id=28c7571632c94d23828dde654e2f54f4, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.441678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.454240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 935/1870] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:39 2022] PUT /identity/v3/projects/a7fe20a198504ec49b892baf0c7dbfbf/users/28c7571632c94d23828dde654e2f54f4/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:39.465397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a8c07-0f8d-41a1-849a-af43b7343247 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.465902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a8c07-0f8d-41a1-849a-af43b7343247 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.466145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a8c07-0f8d-41a1-849a-af43b7343247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.495655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9d9a8c07-0f8d-41a1-849a-af43b7343247 None None] MFA Rules not processed for user `28c7571632c94d23828dde654e2f54f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:39.496869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9d9a8c07-0f8d-41a1-849a-af43b7343247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:39.562473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 936/1871] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:39.567157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.630409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb6db0-ad57-4881-b103-d34b9e8a41d6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.635976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.636605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.636913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.637182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.637815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.639770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.670721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 936/1872] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:39 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:39.674814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a8c07-0f8d-41a1-849a-af43b7343247 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.685287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a8c07-0f8d-41a1-849a-af43b7343247 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.742024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.742637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.742921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.743191 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.743852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.745875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.767695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 937/1873] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:39 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:39.771649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.782396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177a7c4-3ded-4484-bd1c-9ad351bea6e9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.787063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.787654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.787926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.788217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.788854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.790551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.801215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 937/1874] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:39.805295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.815396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d8c04-c682-472c-bbac-94442e7cb82c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.820670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.821335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.821611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.821874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] PATH_INFO: `/v3/projects/5b4e06b0e73746b8931153cb3edcd2a0/users/d45e748144314a17a942dd6c96ef1131/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.838221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4e06b0e73746b8931153cb3edcd2a0, user_id=d45e748144314a17a942dd6c96ef1131, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.840998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.855504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 938/1875] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:39 2022] PUT /identity/v3/projects/5b4e06b0e73746b8931153cb3edcd2a0/users/d45e748144314a17a942dd6c96ef1131/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:39.874466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.885376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f5377-c338-4a7c-b7a5-936320cdf1ff admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:39.961750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa1a75-b591-4b72-aae3-1ee3369cc7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:39.962276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1a75-b591-4b72-aae3-1ee3369cc7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:39.962555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1a75-b591-4b72-aae3-1ee3369cc7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:39.962818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1a75-b591-4b72-aae3-1ee3369cc7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:39.963447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa1a75-b591-4b72-aae3-1ee3369cc7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:39.965083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa1a75-b591-4b72-aae3-1ee3369cc7f2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:39.976301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 938/1876] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:39.979890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:39.992267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db5d8-c063-411e-ae4e-fa38fc360d5a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:40.000304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:40.000994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:40.001270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:40.001539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] PATH_INFO: `/v3/projects/5b4e06b0e73746b8931153cb3edcd2a0/users/d45e748144314a17a942dd6c96ef1131/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:40.003466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4e06b0e73746b8931153cb3edcd2a0, user_id=d45e748144314a17a942dd6c96ef1131, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:40.006142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:40.017453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 939/1877] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:39 2022] PUT /identity/v3/projects/5b4e06b0e73746b8931153cb3edcd2a0/users/d45e748144314a17a942dd6c96ef1131/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:40.029704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161d671-95b6-4fdb-9944-d4c7ea1d6f4b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:40.030241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161d671-95b6-4fdb-9944-d4c7ea1d6f4b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:40.030522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161d671-95b6-4fdb-9944-d4c7ea1d6f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:40.062526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d161d671-95b6-4fdb-9944-d4c7ea1d6f4b None None] MFA Rules not processed for user `d45e748144314a17a942dd6c96ef1131`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:40.063757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d161d671-95b6-4fdb-9944-d4c7ea1d6f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:40.083093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:40.141723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 939/1878] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:25:40.153642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:40.155336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:40.177323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:40.219670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f2374-f244-4906-8b02-369474d691fc admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:40.223456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:40.224069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:40.224376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:40.224648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:40.234261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:40.235681 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:40.246331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 940/1879] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:40.359603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161d671-95b6-4fdb-9944-d4c7ea1d6f4b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:40.370939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161d671-95b6-4fdb-9944-d4c7ea1d6f4b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:40.378534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c0436-bdb1-4521-b9fc-a3ae7234c939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:40.379171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c0436-bdb1-4521-b9fc-a3ae7234c939 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:40.379450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c0436-bdb1-4521-b9fc-a3ae7234c939 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:40.379723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c0436-bdb1-4521-b9fc-a3ae7234c939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:40.456281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c0436-bdb1-4521-b9fc-a3ae7234c939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:40.458238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c0436-bdb1-4521-b9fc-a3ae7234c939 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:40.473262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 940/1880] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:40.839689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:40.910853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc736083-9a98-4b5e-95dd-51e145fec512 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:40.916003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:40.917193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:40.917534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:40.917745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] PATH_INFO: `/v3/users/28c7571632c94d23828dde654e2f54f4` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:40.919009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c7571632c94d23828dde654e2f54f4)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:40.921124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:40.977635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c7571632c94d23828dde654e2f54f4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:40.996989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c7571632c94d23828dde654e2f54f4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:40.997657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c7571632c94d23828dde654e2f54f4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.004853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c7571632c94d23828dde654e2f54f4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.014047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c7571632c94d23828dde654e2f54f4'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.020527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 941/1881] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:40 2022] DELETE /identity/v3/users/28c7571632c94d23828dde654e2f54f4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:41.025263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:41.025769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:41.025864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:41.045459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:41.046752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:41.108516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 941/1882] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:41.118815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:41.120289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:41.168833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:41.187901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387ec7c-95dd-4d2a-8ef0-c9a6c48d30d7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:41.192727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:41.193468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:41.193932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:41.194421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:41.204945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:41.206461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:41.219004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 942/1883] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:25:41.535816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:41.537110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:41.559464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0e1d-6c14-413f-980d-6bc94af52ae5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:41.574923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:41.575801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:41.576823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:41.576938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] PATH_INFO: `/v3/projects/a7fe20a198504ec49b892baf0c7dbfbf` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:41.583970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7fe20a198504ec49b892baf0c7dbfbf)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:41.589855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:41.668613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7fe20a198504ec49b892baf0c7dbfbf', 'request_id': 'req-47a8662b-f399-45b2-a690-db5b9c3ab5ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.670697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:41.683261 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7fe20a198504ec49b892baf0c7dbfbf', 'request_id': 'req-47a8662b-f399-45b2-a690-db5b9c3ab5ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.692426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7fe20a198504ec49b892baf0c7dbfbf', 'request_id': 'req-47a8662b-f399-45b2-a690-db5b9c3ab5ca'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.709685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] The token cache is being invalidate because project a7fe20a198504ec49b892baf0c7dbfbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:41.709685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.710238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.712128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 942/1884] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:41 2022] DELETE /identity/v3/projects/a7fe20a198504ec49b892baf0c7dbfbf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:41.715985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:41.731049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a77aa-f7c9-49a8-aa26-0594346fbfa7 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:41.732128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:41.734432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:41.848074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8662b-f399-45b2-a690-db5b9c3ab5ca admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:41.853534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:41.854066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:41.854066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:41.854219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] PATH_INFO: `/v3/users/d45e748144314a17a942dd6c96ef1131` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:41.858321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d45e748144314a17a942dd6c96ef1131)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:41.858809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:41.859465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:41.859738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:41.860005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] PATH_INFO: `/v3/users/85d4be5477d54a2bba6cfb0a7e59c4b5` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:41.860284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:41.862070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d4be5477d54a2bba6cfb0a7e59c4b5)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:41.863991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:41.947862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd45e748144314a17a942dd6c96ef1131'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.958940 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d4be5477d54a2bba6cfb0a7e59c4b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.965256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd45e748144314a17a942dd6c96ef1131'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.966533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd45e748144314a17a942dd6c96ef1131'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.980430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd45e748144314a17a942dd6c96ef1131'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.983916 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d4be5477d54a2bba6cfb0a7e59c4b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.984979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d4be5477d54a2bba6cfb0a7e59c4b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:41.993343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e2fd8e76-ab93-4214-a067-16f41fa09dd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd45e748144314a17a942dd6c96ef1131'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.011242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 943/1885] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:41 2022] DELETE /identity/v3/users/d45e748144314a17a942dd6c96ef1131 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:42.012653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d4be5477d54a2bba6cfb0a7e59c4b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.035847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d4be5477d54a2bba6cfb0a7e59c4b5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.044696 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 943/1886] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:41 2022] DELETE /identity/v3/users/85d4be5477d54a2bba6cfb0a7e59c4b5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:42.048992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:42.049296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:42.049565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:42.075077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:42.076312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:42.136935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:42.151842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 944/1887] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:42.175015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:42.176514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:42.241122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfde219-b49d-4059-8e9f-46fcc0f35ee5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:42.246161 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c33ba2-8100-416c-aec8-9549b3be3ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:42.247185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c33ba2-8100-416c-aec8-9549b3be3ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:42.247463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c33ba2-8100-416c-aec8-9549b3be3ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:42.247741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c33ba2-8100-416c-aec8-9549b3be3ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:42.262786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c33ba2-8100-416c-aec8-9549b3be3ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:42.264494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c33ba2-8100-416c-aec8-9549b3be3ce0 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:42.269294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:42.278383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 944/1888] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:42.740719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:42.743156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:42.758246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02f073-d584-4ada-8242-4ac1bf5620f3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:42.764684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:42.765868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:42.766160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:42.766836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] PATH_INFO: `/v3/projects/5b4e06b0e73746b8931153cb3edcd2a0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:42.768110 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4e06b0e73746b8931153cb3edcd2a0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:42.770693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:42.830728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4e06b0e73746b8931153cb3edcd2a0', 'request_id': 'req-d9b36566-5a7e-484f-af17-639eedf2699d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.841356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4e06b0e73746b8931153cb3edcd2a0', 'request_id': 'req-d9b36566-5a7e-484f-af17-639eedf2699d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.850763 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4e06b0e73746b8931153cb3edcd2a0', 'request_id': 'req-d9b36566-5a7e-484f-af17-639eedf2699d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.865448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] The token cache is being invalidate because project 5b4e06b0e73746b8931153cb3edcd2a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:42.865448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.865448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:42.867259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 945/1889] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:42 2022] DELETE /identity/v3/projects/5b4e06b0e73746b8931153cb3edcd2a0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:42.872500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c4678-1365-44b7-867a-34229a2e3f1f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:42.873008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c4678-1365-44b7-867a-34229a2e3f1f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:42.873460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c4678-1365-44b7-867a-34229a2e3f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:42.924303 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ca6c4678-1365-44b7-867a-34229a2e3f1f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:42.925805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ca6c4678-1365-44b7-867a-34229a2e3f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:42.936687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:42.939290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:43.067276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 945/1890] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:43.070546 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90aca0-0043-4f53-aea4-0e5b007d7363 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.071037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90aca0-0043-4f53-aea4-0e5b007d7363 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.071501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90aca0-0043-4f53-aea4-0e5b007d7363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.088674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36566-5a7e-484f-af17-639eedf2699d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.093963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7a90aca0-0043-4f53-aea4-0e5b007d7363 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:43.095346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7a90aca0-0043-4f53-aea4-0e5b007d7363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:43.096622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.097307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.097690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.098066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] PATH_INFO: `/v3/projects/cc29c7be9aff419f97301a95eab7755e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.099321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc29c7be9aff419f97301a95eab7755e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.101564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.118600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 946/1891] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:43.119710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90aca0-0043-4f53-aea4-0e5b007d7363 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:43.131649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90aca0-0043-4f53-aea4-0e5b007d7363 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.139184 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.140256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.141964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.141964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.142460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:43.142761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.144907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.166008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 947/1892] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:43.170558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:43.183549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eec916-d52d-43ba-8ba4-2dd578041b0e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.189362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.190351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.190743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.191131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.191827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.193945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.205543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc29c7be9aff419f97301a95eab7755e', 'request_id': 'req-a2a7cd76-2303-4d02-b7a7-55d56d9add04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:43.219443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc29c7be9aff419f97301a95eab7755e', 'request_id': 'req-a2a7cd76-2303-4d02-b7a7-55d56d9add04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:43.229287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc29c7be9aff419f97301a95eab7755e', 'request_id': 'req-a2a7cd76-2303-4d02-b7a7-55d56d9add04'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:43.237971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 948/1893] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:43 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:43.239080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:43.248612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] The token cache is being invalidate because project cc29c7be9aff419f97301a95eab7755e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:43.249303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:43.250829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:43.253270 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 946/1894] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:42 2022] DELETE /identity/v3/projects/cc29c7be9aff419f97301a95eab7755e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:43.367078 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e5fd5-efc9-4f64-902b-67161a54c078 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.379494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.380350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.380727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.381095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.382040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.385249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.431835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 949/1895] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:43 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:43.437465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:43.441900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:43.476506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:43.536366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7cd76-2303-4d02-b7a7-55d56d9add04 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.542028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.542875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.543327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.543905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.544668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.546258 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.557395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 947/1896] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:43.561402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:43.571579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23599548-fb5d-428f-ab85-00b8031bfe4d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.576213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa2d2-0bb7-4496-a7a4-99cf5d517ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.576852 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa2d2-0bb7-4496-a7a4-99cf5d517ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.577151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa2d2-0bb7-4496-a7a4-99cf5d517ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.577419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa2d2-0bb7-4496-a7a4-99cf5d517ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.578058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa2d2-0bb7-4496-a7a4-99cf5d517ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.579655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa2d2-0bb7-4496-a7a4-99cf5d517ce2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.589830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 950/1897] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:25:43.593712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:43.604642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca787c3-8476-47cd-95c3-fc13a9890c1f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.609103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.609778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.610051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.610320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] PATH_INFO: `/v3/projects/ad61050683b747ffa68e611d740a77fe/users/36614edf3ab84279bf3ef86f00a2af4d/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.631794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad61050683b747ffa68e611d740a77fe, user_id=36614edf3ab84279bf3ef86f00a2af4d, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.634797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.648161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 948/1898] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:43 2022] PUT /identity/v3/projects/ad61050683b747ffa68e611d740a77fe/users/36614edf3ab84279bf3ef86f00a2af4d/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:43.659775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce45d3-f149-4e92-b0d8-ef2761210705 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.660053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce45d3-f149-4e92-b0d8-ef2761210705 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.660374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce45d3-f149-4e92-b0d8-ef2761210705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.692518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-42ce45d3-f149-4e92-b0d8-ef2761210705 None None] MFA Rules not processed for user `36614edf3ab84279bf3ef86f00a2af4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:43.693727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-42ce45d3-f149-4e92-b0d8-ef2761210705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:43.719224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:43.764357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 951/1899] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:43.776380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:43.778073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:43.835318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006d24-afec-4f6e-87af-d73afbaaa1e6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:43.838940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78e9a0-9d7c-4445-bd8d-eb1b0a5ea574 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:43.839543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78e9a0-9d7c-4445-bd8d-eb1b0a5ea574 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:43.839819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78e9a0-9d7c-4445-bd8d-eb1b0a5ea574 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:43.840088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78e9a0-9d7c-4445-bd8d-eb1b0a5ea574 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:43.848549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78e9a0-9d7c-4445-bd8d-eb1b0a5ea574 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:43.849966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78e9a0-9d7c-4445-bd8d-eb1b0a5ea574 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:43.859074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 949/1900] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:46.418000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce45d3-f149-4e92-b0d8-ef2761210705 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:46.478041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce45d3-f149-4e92-b0d8-ef2761210705 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:46.482617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:46.483254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:46.483544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:46.483789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] PATH_INFO: `/v3/users/36614edf3ab84279bf3ef86f00a2af4d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:46.484923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] RBAC: Authorizing `identity:delete_user(user_id=36614edf3ab84279bf3ef86f00a2af4d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:46.486814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:46.545863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36614edf3ab84279bf3ef86f00a2af4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:46.560138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36614edf3ab84279bf3ef86f00a2af4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:46.561373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36614edf3ab84279bf3ef86f00a2af4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:46.569253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36614edf3ab84279bf3ef86f00a2af4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:46.579907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36614edf3ab84279bf3ef86f00a2af4d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:46.606892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 952/1901] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:25:46 2022] DELETE /identity/v3/users/36614edf3ab84279bf3ef86f00a2af4d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:46.618111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:46.618591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:46.618870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:46.654659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:46.656285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:46.732590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 950/1902] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:46.749545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:46.753724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:46.826328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:46.839162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b672de-8979-48de-9c7a-299cbacdf20d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:46.842905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc0734-e607-4563-b9ff-25c2200a323b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:46.843880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc0734-e607-4563-b9ff-25c2200a323b service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:46.844396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc0734-e607-4563-b9ff-25c2200a323b service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:46.844668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc0734-e607-4563-b9ff-25c2200a323b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:46.854578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc0734-e607-4563-b9ff-25c2200a323b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:46.856082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc0734-e607-4563-b9ff-25c2200a323b service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:46.867080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 953/1903] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:47.189320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.190776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:47.228427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30388f01-b0be-4e0f-bdb6-ad1c26a670ac None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:47.234388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:47.235057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.235410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.235739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] PATH_INFO: `/v3/projects/ad61050683b747ffa68e611d740a77fe` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.237545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad61050683b747ffa68e611d740a77fe)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:47.251345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:47.292455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:47.324088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad61050683b747ffa68e611d740a77fe', 'request_id': 'req-4d34e45c-e9f3-47f0-84ef-5479b0695510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:47.332825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad61050683b747ffa68e611d740a77fe', 'request_id': 'req-4d34e45c-e9f3-47f0-84ef-5479b0695510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:47.342435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad61050683b747ffa68e611d740a77fe', 'request_id': 'req-4d34e45c-e9f3-47f0-84ef-5479b0695510'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:47.355526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] The token cache is being invalidate because project ad61050683b747ffa68e611d740a77fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:25:47.355817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:47.356500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4d34e45c-e9f3-47f0-84ef-5479b0695510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:25:47.361279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 951/1904] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:25:47 2022] DELETE /identity/v3/projects/ad61050683b747ffa68e611d740a77fe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:25:47.362959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aac60-e918-498d-98ff-057484a05cf0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.363292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aac60-e918-498d-98ff-057484a05cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.363569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aac60-e918-498d-98ff-057484a05cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.383579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-820aac60-e918-498d-98ff-057484a05cf0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:47.385080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-820aac60-e918-498d-98ff-057484a05cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:47.454307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 954/1905] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:47.462536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298a957-89be-42db-a723-9acd06f871f5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.462889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298a957-89be-42db-a723-9acd06f871f5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.463244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298a957-89be-42db-a723-9acd06f871f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.482660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d298a957-89be-42db-a723-9acd06f871f5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:47.483932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d298a957-89be-42db-a723-9acd06f871f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:47.493754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 952/1906] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:47.497834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820aac60-e918-498d-98ff-057484a05cf0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.508283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820aac60-e918-498d-98ff-057484a05cf0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:47.513187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:47.513791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.514065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.514330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.514965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:25:47.515264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:47.516875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:47.528964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 955/1907] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:25:47.533369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298a957-89be-42db-a723-9acd06f871f5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.543532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298a957-89be-42db-a723-9acd06f871f5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:47.548146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:47.548784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.549061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.549328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.549969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:47.551887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:47.580738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 953/1908] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:25:47 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:25:47.584934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.595876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e80c38-9165-42b6-9e6c-5c459daa3be4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:47.659850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:47.660428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.660704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.660974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.661606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:47.664059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:47.683956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 956/1909] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:25:47 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:25:47.688146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.698754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ace78-5e2e-4dcd-90e1-5a830008bb55 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:47.703246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:47.703829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.704105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.704406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.705031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:47.706612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:47.716199 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 954/1910] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:47.719993 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.729674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703f443-cc66-4d40-bfda-209dd8fa4c82 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:47.734158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e25fea-9452-4448-8f98-132b9c2fbd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:47.734744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e25fea-9452-4448-8f98-132b9c2fbd4d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.735020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e25fea-9452-4448-8f98-132b9c2fbd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.735286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e25fea-9452-4448-8f98-132b9c2fbd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.735915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e25fea-9452-4448-8f98-132b9c2fbd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:47.737546 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e25fea-9452-4448-8f98-132b9c2fbd4d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:47.748690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 957/1911] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:25:47.752474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.763116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e613d-757d-4384-9c1e-3d13526ace57 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:47.767663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:47.768342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.768618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.768889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] PATH_INFO: `/v3/projects/fc06c8b6218344079b29ecdcc2c05056/users/2bc346316c134db096dc01a5e1d48696/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.783024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc06c8b6218344079b29ecdcc2c05056, user_id=2bc346316c134db096dc01a5e1d48696, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:47.785665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:47.798042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 955/1912] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:25:47 2022] PUT /identity/v3/projects/fc06c8b6218344079b29ecdcc2c05056/users/2bc346316c134db096dc01a5e1d48696/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:25:47.808820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c33866-3116-4e5c-8570-f734532e3466 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:47.809121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c33866-3116-4e5c-8570-f734532e3466 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:47.809390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c33866-3116-4e5c-8570-f734532e3466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:47.836610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-66c33866-3116-4e5c-8570-f734532e3466 None None] MFA Rules not processed for user `2bc346316c134db096dc01a5e1d48696`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:25:47.837803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-66c33866-3116-4e5c-8570-f734532e3466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:47.880423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:47.920938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 958/1913] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:25:47.933008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:25:47.934938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:25:48.001929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33b6c5-4d22-4f25-971e-0e108ddbb2ff admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:25:48.006311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:25:48.007382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:25:48.007673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:25:48.008071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:25:48.020131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:25:48.021727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:25:48.032720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 956/1914] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:25:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:25:57.918050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=8 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:25:58.028401 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:03.310534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c606-b099-43a9-959d-b0ae19fcc6cb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:03.311080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c606-b099-43a9-959d-b0ae19fcc6cb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:03.311080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c606-b099-43a9-959d-b0ae19fcc6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:03.330092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-dfb6c606-b099-43a9-959d-b0ae19fcc6cb None None] MFA Rules not processed for user `97d696ea0031417d8a1a0b8471c26579`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:03.331304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-dfb6c606-b099-43a9-959d-b0ae19fcc6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:03.400426 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 959/1915] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:06.891517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:06.893000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:06.989888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae950a-96b7-4755-a5f7-e46009f7d355 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:06.995471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:06.995471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:06.995806 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:06.995990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] PATH_INFO: `/v3/users/5d0168d1b0a34711af06bdbf9ae19a6e` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:06.997812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0168d1b0a34711af06bdbf9ae19a6e)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:06.999888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:07.057044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0168d1b0a34711af06bdbf9ae19a6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.077722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0168d1b0a34711af06bdbf9ae19a6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.078666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0168d1b0a34711af06bdbf9ae19a6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.085763 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0168d1b0a34711af06bdbf9ae19a6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.102279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0168d1b0a34711af06bdbf9ae19a6e'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.108764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 957/1916] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:26:06 2022] DELETE /identity/v3/users/5d0168d1b0a34711af06bdbf9ae19a6e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:07.113277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:07.113757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:07.113874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:07.134432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:07.135541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:07.196525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 960/1917] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:07.207513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:07.209397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:07.290602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc396e-69aa-4dd4-bc99-6dc851433764 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:07.297747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74fbde-8d28-4f55-8638-6e80be30c3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:07.298675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74fbde-8d28-4f55-8638-6e80be30c3fe service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:07.299112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74fbde-8d28-4f55-8638-6e80be30c3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:07.299540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74fbde-8d28-4f55-8638-6e80be30c3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:07.313821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74fbde-8d28-4f55-8638-6e80be30c3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:07.316157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74fbde-8d28-4f55-8638-6e80be30c3fe service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:07.328779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 958/1918] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:07.653895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:07.655435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:07.668559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee9101-1b07-4a9d-ba6a-9e348d30d034 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:07.673428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:07.674062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:07.674337 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:07.675333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] PATH_INFO: `/v3/projects/0b9364d7587d4041beadcab24aebb3db` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:07.676825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9364d7587d4041beadcab24aebb3db)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:07.679362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:07.731177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9364d7587d4041beadcab24aebb3db', 'request_id': 'req-d622652b-0cdc-40bd-9d3c-74663f50f15f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.738138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9364d7587d4041beadcab24aebb3db', 'request_id': 'req-d622652b-0cdc-40bd-9d3c-74663f50f15f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.745058 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9364d7587d4041beadcab24aebb3db', 'request_id': 'req-d622652b-0cdc-40bd-9d3c-74663f50f15f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.756935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] The token cache is being invalidate because project 0b9364d7587d4041beadcab24aebb3db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:26:07.757221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.758993 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d622652b-0cdc-40bd-9d3c-74663f50f15f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:07.760423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 961/1919] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:26:07 2022] DELETE /identity/v3/projects/0b9364d7587d4041beadcab24aebb3db => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:07.765615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04abf6-ecb9-40cb-830f-2d29173522f4 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:07.765900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04abf6-ecb9-40cb-830f-2d29173522f4 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:07.766175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04abf6-ecb9-40cb-830f-2d29173522f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:07.786204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bf04abf6-ecb9-40cb-830f-2d29173522f4 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:07.787509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bf04abf6-ecb9-40cb-830f-2d29173522f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:07.805814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:07.858093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 959/1920] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:07.866022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852547b-ee00-422d-9160-281b8164311f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:07.866298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852547b-ee00-422d-9160-281b8164311f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:07.866562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852547b-ee00-422d-9160-281b8164311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:07.886285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c852547b-ee00-422d-9160-281b8164311f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:07.887494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c852547b-ee00-422d-9160-281b8164311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:07.895232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 962/1921] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:07.900903 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04abf6-ecb9-40cb-830f-2d29173522f4 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:07.914447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04abf6-ecb9-40cb-830f-2d29173522f4 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:07.919114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:07.919690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:07.919964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:07.920248 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:07.920910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:26:07.921207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:07.922807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:07.934303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 960/1922] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:26:07.938659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852547b-ee00-422d-9160-281b8164311f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:07.947832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852547b-ee00-422d-9160-281b8164311f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:07.952480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:07.953081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:07.953359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:07.953630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:07.954307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:07.956173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:07.978531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:07.985536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 963/1923] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:26:07 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:26:07.989541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:08.000057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85fec9-fb8e-4d16-a004-88eec290885a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:08.068587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:08.069504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:08.070239 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:08.070475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:08.071255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:08.073272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:08.098866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 961/1924] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:26:07 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:26:08.100747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:08.112317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b6dfd-5f20-4236-92b1-2fe278dbb8c0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:08.120687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:08.121443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:08.121724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:08.121994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:08.122636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:08.124254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:08.136404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 964/1925] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:26:08.140695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:08.155250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddf659-b4ec-4860-8fc1-81b7f43ef640 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:08.160287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3150306-83c0-499a-a997-e21c0f181354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:08.160987 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3150306-83c0-499a-a997-e21c0f181354 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:08.161334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3150306-83c0-499a-a997-e21c0f181354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:08.161677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3150306-83c0-499a-a997-e21c0f181354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:08.162328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3150306-83c0-499a-a997-e21c0f181354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:08.163947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3150306-83c0-499a-a997-e21c0f181354 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:08.176008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 962/1926] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:26:08.180159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:08.192591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbed3e2-4f4f-4a9e-a72f-8c7cda73fd2d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:08.198815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:08.199487 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:08.199766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:08.200035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] PATH_INFO: `/v3/projects/412ef1dbb71f4d62956608eb8d404937/users/d1d126a141f540c3b1600cb195640b6c/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:08.217639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] RBAC: Authorizing `identity:create_grant(project_id=412ef1dbb71f4d62956608eb8d404937, user_id=d1d126a141f540c3b1600cb195640b6c, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:08.220335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:08.235711 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 965/1927] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:26:08 2022] PUT /identity/v3/projects/412ef1dbb71f4d62956608eb8d404937/users/d1d126a141f540c3b1600cb195640b6c/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:08.247699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:08.247989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:08.248295 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:08.280120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] MFA Rules not processed for user `d1d126a141f540c3b1600cb195640b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:08.281747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:08.360279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 963/1928] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:08.374172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:08.376985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:08.526749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a1a8-01fa-4058-a237-66b56600835e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:08.531550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a9649-c7bf-49ab-8b35-8d964aa7b222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:08.535000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a9649-c7bf-49ab-8b35-8d964aa7b222 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:08.535338 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a9649-c7bf-49ab-8b35-8d964aa7b222 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:08.535781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a9649-c7bf-49ab-8b35-8d964aa7b222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:08.547901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a9649-c7bf-49ab-8b35-8d964aa7b222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:08.549400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a9649-c7bf-49ab-8b35-8d964aa7b222 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:08.561344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 966/1929] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:11.047286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:11.051017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:11.075424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:11.153427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03931aa-c78d-4973-bada-ac218f9d36c3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:11.160739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:11.161469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:11.161890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:11.162881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] PATH_INFO: `/v3/users/2bc346316c134db096dc01a5e1d48696` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:11.165262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc346316c134db096dc01a5e1d48696)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:11.168002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:11.222538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc346316c134db096dc01a5e1d48696'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.235470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc346316c134db096dc01a5e1d48696'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.237315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc346316c134db096dc01a5e1d48696'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.244441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc346316c134db096dc01a5e1d48696'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.252939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc346316c134db096dc01a5e1d48696'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.261652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 964/1930] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:26:11 2022] DELETE /identity/v3/users/2bc346316c134db096dc01a5e1d48696 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:11.266976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c2a8-6893-4ec7-afb3-c616ea35f748 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:11.268358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c2a8-6893-4ec7-afb3-c616ea35f748 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:11.268358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0c2a8-6893-4ec7-afb3-c616ea35f748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:11.287485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c7e0c2a8-6893-4ec7-afb3-c616ea35f748 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:11.288781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c7e0c2a8-6893-4ec7-afb3-c616ea35f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:11.349266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 967/1931] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:11.358596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:11.360000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:11.419334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67be427-ace4-4d20-b78b-4d442beef71e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:11.423025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:11.423633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:11.423904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:11.424169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:11.436024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:11.437478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:11.447765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 965/1932] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:11.611111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:11.611388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:11.611655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:11.631425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:11.632712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:11.696172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 968/1933] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:26:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:11.736902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:11.738344 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:11.789951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:11.817742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0272cf-39f6-49a6-a28d-97dc2186052f service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:11.824543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:11.826019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:11.826795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cdbf4-02cd-43bf-b2bc-30de4c67494b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:11.827520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cdbf4-02cd-43bf-b2bc-30de4c67494b service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:11.827947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cdbf4-02cd-43bf-b2bc-30de4c67494b service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:11.828343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cdbf4-02cd-43bf-b2bc-30de4c67494b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:11.838892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cdbf4-02cd-43bf-b2bc-30de4c67494b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:11.840602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cdbf4-02cd-43bf-b2bc-30de4c67494b service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:11.846748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf58e9-d354-4049-be05-7ee35e73a68c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:11.852501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:11.853132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:11.853475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:11.853658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] PATH_INFO: `/v3/projects/fc06c8b6218344079b29ecdcc2c05056` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:11.854824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc06c8b6218344079b29ecdcc2c05056)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:11.856741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:11.860443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 966/1934] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:26:11.865867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:11.918898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc06c8b6218344079b29ecdcc2c05056', 'request_id': 'req-668a0545-069c-4090-9e8c-f38a65840c1d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.930380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc06c8b6218344079b29ecdcc2c05056', 'request_id': 'req-668a0545-069c-4090-9e8c-f38a65840c1d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.938526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc06c8b6218344079b29ecdcc2c05056', 'request_id': 'req-668a0545-069c-4090-9e8c-f38a65840c1d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.956461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] The token cache is being invalidate because project fc06c8b6218344079b29ecdcc2c05056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:26:11.956887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.957724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:11.960665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 969/1935] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:26:11 2022] DELETE /identity/v3/projects/fc06c8b6218344079b29ecdcc2c05056 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:26:12.543916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759666a-9ad5-43f3-86d2-5441f2b44055 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:12.544394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759666a-9ad5-43f3-86d2-5441f2b44055 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:12.544484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759666a-9ad5-43f3-86d2-5441f2b44055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:12.564047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-9759666a-9ad5-43f3-86d2-5441f2b44055 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:12.565242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9759666a-9ad5-43f3-86d2-5441f2b44055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:12.620061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 967/1936] 10.222.0.48 () {58 vars in 967 bytes} [Thu Oct 27 13:26:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:12.633726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:12.635099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:12.704666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a0545-069c-4090-9e8c-f38a65840c1d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:12.709787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:12.710433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:12.710708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:12.710979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:12.719912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:12.721553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:12.731551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 970/1937] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:26:13.004182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38438-f629-4d76-aba1-e85661e538f7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:13.004502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38438-f629-4d76-aba1-e85661e538f7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:13.004777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38438-f629-4d76-aba1-e85661e538f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:13.027864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f3e38438-f629-4d76-aba1-e85661e538f7 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:13.029086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f3e38438-f629-4d76-aba1-e85661e538f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:13.036490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 968/1938] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:26:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:13.054932 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:13.065151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865e97e-a347-47db-9d92-adf9ab26a7fa service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:13.070794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:13.071404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:13.071680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:13.071945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:13.080434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:13.082033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:13.090882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 971/1939] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:26:13.564194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:13.564694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:13.564809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:13.583716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:13.585085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:13.604392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 969/1940] 10.222.0.48 () {58 vars in 967 bytes} [Thu Oct 27 13:26:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:13.622579 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:13.634811 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a479b-1c01-48a7-82fc-57666a162184 service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:13.639806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e85fe9-7b9d-4d37-8914-50302bb3acfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:13.640409 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85fe9-7b9d-4d37-8914-50302bb3acfc service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:13.640681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85fe9-7b9d-4d37-8914-50302bb3acfc service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:13.641008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85fe9-7b9d-4d37-8914-50302bb3acfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:13.651625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e85fe9-7b9d-4d37-8914-50302bb3acfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:13.654538 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e85fe9-7b9d-4d37-8914-50302bb3acfc service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:13.666482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 972/1941] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:26:16.140357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:16.142474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:16.224220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395d4ba-f3d5-4265-b1a4-fd4ba9e7206c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:16.229454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:16.230089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:16.230453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:16.230654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] PATH_INFO: `/v3/users/e89e34bc3817448d93d576348bd346de` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:16.231768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89e34bc3817448d93d576348bd346de)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:16.233712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:16.283935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89e34bc3817448d93d576348bd346de'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.297094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:16.298981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89e34bc3817448d93d576348bd346de'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.299813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89e34bc3817448d93d576348bd346de'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.307949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89e34bc3817448d93d576348bd346de'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.316459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89e34bc3817448d93d576348bd346de'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.322875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 970/1942] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:26:16 2022] DELETE /identity/v3/users/e89e34bc3817448d93d576348bd346de => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:16.327312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:16.327782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:16.327985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:16.347757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:16.349373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:16.402731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:16.415386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 973/1943] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:16.422227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:16.423728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:16.491756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e348a-9aff-48ff-9b4e-1f54c5e5e938 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:16.495584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1a877-2ad4-4bb6-bf44-bc3625186b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:16.496214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1a877-2ad4-4bb6-bf44-bc3625186b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:16.496530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1a877-2ad4-4bb6-bf44-bc3625186b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:16.496801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1a877-2ad4-4bb6-bf44-bc3625186b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:16.507166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1a877-2ad4-4bb6-bf44-bc3625186b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:16.508888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1a877-2ad4-4bb6-bf44-bc3625186b58 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:16.518902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 971/1944] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:16.712916 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:16.715873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:16.729443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c312-8175-47b2-bf0b-1797a15c81f9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:16.735010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:16.735778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:16.736121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:16.736475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] PATH_INFO: `/v3/projects/ad0ac3d8983248bcad9b84c1be67bc47` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:16.737663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0ac3d8983248bcad9b84c1be67bc47)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:16.739580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:16.810300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0ac3d8983248bcad9b84c1be67bc47', 'request_id': 'req-3ea85440-ed57-4ee0-b25f-3f1611c1b665'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.822379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0ac3d8983248bcad9b84c1be67bc47', 'request_id': 'req-3ea85440-ed57-4ee0-b25f-3f1611c1b665'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.829326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0ac3d8983248bcad9b84c1be67bc47', 'request_id': 'req-3ea85440-ed57-4ee0-b25f-3f1611c1b665'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.843347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] The token cache is being invalidate because project ad0ac3d8983248bcad9b84c1be67bc47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:26:16.843639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.845113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3ea85440-ed57-4ee0-b25f-3f1611c1b665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:16.846944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 974/1945] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:26:16 2022] DELETE /identity/v3/projects/ad0ac3d8983248bcad9b84c1be67bc47 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:16.852833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c60281-3db0-4bae-88b3-753778ba6a7d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:16.853114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c60281-3db0-4bae-88b3-753778ba6a7d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:16.853381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c60281-3db0-4bae-88b3-753778ba6a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:16.874439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-15c60281-3db0-4bae-88b3-753778ba6a7d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:16.875730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-15c60281-3db0-4bae-88b3-753778ba6a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:16.955953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 972/1946] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:17.027646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186f26-241b-41cf-8658-ea0b0d8d74c2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.028210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186f26-241b-41cf-8658-ea0b0d8d74c2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.028817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186f26-241b-41cf-8658-ea0b0d8d74c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.059927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7e186f26-241b-41cf-8658-ea0b0d8d74c2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:17.062148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7e186f26-241b-41cf-8658-ea0b0d8d74c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:17.074401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 975/1947] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:17.077139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c60281-3db0-4bae-88b3-753778ba6a7d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:17.088524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c60281-3db0-4bae-88b3-753778ba6a7d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:17.093810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:17.094430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.094708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.094978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.095644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:26:17.095943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:17.097608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:17.116081 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 973/1948] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:26:17.117070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186f26-241b-41cf-8658-ea0b0d8d74c2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:17.128864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186f26-241b-41cf-8658-ea0b0d8d74c2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:17.138241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:17.139070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.139238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.139525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.140166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:17.142517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:17.195506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 976/1949] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:26:17 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:26:17.199796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:17.210829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39ecf-fa65-46fd-b36b-c0e7aa05caf5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:17.284697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:17.285312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.285591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.285864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.286641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:17.288631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:17.308041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:17.313766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 974/1950] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:26:17 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:26:17.318163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:17.329324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6db41-e087-4443-b4ce-20af4e5b2f6f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:17.334657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:17.335335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.335702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.336041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.336854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:17.338771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:17.353847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 977/1951] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:26:17.355257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:17.365449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0360e-c8b3-41af-8754-cc36c68efdce admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:17.370527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a5282-851b-454b-b4f9-49ce0c2f9412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:17.371238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a5282-851b-454b-b4f9-49ce0c2f9412 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.371519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a5282-851b-454b-b4f9-49ce0c2f9412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.371789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a5282-851b-454b-b4f9-49ce0c2f9412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.372461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a5282-851b-454b-b4f9-49ce0c2f9412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:17.374056 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a5282-851b-454b-b4f9-49ce0c2f9412 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:17.386600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 975/1952] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:26:17.390946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:17.401960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3353-6858-4d3a-8de9-ef91a1187d0f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:17.406757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:17.407416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.408358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.408632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] PATH_INFO: `/v3/projects/52552d69672a4677bd52c847cceec247/users/706ba757aacf4256b1416b5e3bbda714/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.425991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52552d69672a4677bd52c847cceec247, user_id=706ba757aacf4256b1416b5e3bbda714, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:17.429240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:17.442412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 978/1953] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:26:17 2022] PUT /identity/v3/projects/52552d69672a4677bd52c847cceec247/users/706ba757aacf4256b1416b5e3bbda714/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:17.447642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58f6b4-7318-43ba-bc4d-1a5c605ee962 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.447923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58f6b4-7318-43ba-bc4d-1a5c605ee962 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.448192 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58f6b4-7318-43ba-bc4d-1a5c605ee962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.467895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8f58f6b4-7318-43ba-bc4d-1a5c605ee962 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:17.469107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8f58f6b4-7318-43ba-bc4d-1a5c605ee962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:17.530288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 976/1954] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:17.540755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:17.542449 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:17.597156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced44a66-c75b-440f-94d6-f2e7321a7575 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:17.601421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:17.602022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:17.602293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:17.602556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:17.611332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:17.612818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:17.622197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 979/1955] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:21.409531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce64516-335d-4f0a-8faa-4c7101397686 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:21.410032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce64516-335d-4f0a-8faa-4c7101397686 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:21.410032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce64516-335d-4f0a-8faa-4c7101397686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:21.454697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-cce64516-335d-4f0a-8faa-4c7101397686 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:21.456031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-cce64516-335d-4f0a-8faa-4c7101397686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:21.602465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 977/1956] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:26:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:21.610266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:21.611818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:21.648739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:21.736575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b541e6-d701-4e21-9250-e3fc164f7a53 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:21.743785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:21.752412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:21.753018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:21.753380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:21.769604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:21.771872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:21.788377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 980/1957] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:26:23.477273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136d88a-38cf-49d0-b00c-f9d5562d3026 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:23.478143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136d88a-38cf-49d0-b00c-f9d5562d3026 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:23.478578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136d88a-38cf-49d0-b00c-f9d5562d3026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:23.523765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b136d88a-38cf-49d0-b00c-f9d5562d3026 None None] MFA Rules not processed for user `706ba757aacf4256b1416b5e3bbda714`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:23.525462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b136d88a-38cf-49d0-b00c-f9d5562d3026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:23.672317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 978/1958] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:23.688580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:23.722411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437c626-5248-426e-88fb-6af2886935fd service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:23.730091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:23.730870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:23.731228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:23.731748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:23.733284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:23.735453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:23.796532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 981/1959] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:26:23 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:26:23.797678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136d88a-38cf-49d0-b00c-f9d5562d3026 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:23.834027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136d88a-38cf-49d0-b00c-f9d5562d3026 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:23.928964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:23.996739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:23.997382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:23.997664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:23.998135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:23.998796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:24.001294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:24.041305 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 979/1960] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:26:23 2022] POST /identity/v3/users => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:26:24.042626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:24.058318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77277a10-5d5f-48b0-b3bd-f2eca27af8de admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:24.080742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:24.081372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:24.081656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:24.081922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:24.082832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:24.084291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:24.130937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 982/1961] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:26:24.135453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:24.149678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7af2f9-8709-4a24-a3ba-74d3bfeb760a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:24.159918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:24.162780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:24.163064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:24.163350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] PATH_INFO: `/v3/projects/9daea5edf5364f639b31d351f960bcc1/users/dc3bac1dedef4f52a889f4835729c000/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:24.204299 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9daea5edf5364f639b31d351f960bcc1, user_id=dc3bac1dedef4f52a889f4835729c000, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:24.207047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:24.242195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 980/1962] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:26:24 2022] PUT /identity/v3/projects/9daea5edf5364f639b31d351f960bcc1/users/dc3bac1dedef4f52a889f4835729c000/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:24.245969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:24.259766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f980e1-e35d-4386-bffb-d08789121d35 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:24.437533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec95124-ce99-470c-9443-1caddad93969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:24.438370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec95124-ce99-470c-9443-1caddad93969 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:24.439053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec95124-ce99-470c-9443-1caddad93969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:24.439053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec95124-ce99-470c-9443-1caddad93969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:24.439834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec95124-ce99-470c-9443-1caddad93969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:24.441673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec95124-ce99-470c-9443-1caddad93969 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:24.461409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:24.468867 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 983/1963] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:26:24.475584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022fc15-459b-4e76-9c52-dcb4af5c5c5d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:24.481469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:24.482141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:24.482423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:24.482697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] PATH_INFO: `/v3/projects/9daea5edf5364f639b31d351f960bcc1/users/dc3bac1dedef4f52a889f4835729c000/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:24.485039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9daea5edf5364f639b31d351f960bcc1, user_id=dc3bac1dedef4f52a889f4835729c000, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:24.487883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:24.504288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 981/1964] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:26:24 2022] PUT /identity/v3/projects/9daea5edf5364f639b31d351f960bcc1/users/dc3bac1dedef4f52a889f4835729c000/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:32.295067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e642fb-10a3-4d6e-ac88-eb6407624cf0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:32.295976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e642fb-10a3-4d6e-ac88-eb6407624cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:32.296432 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e642fb-10a3-4d6e-ac88-eb6407624cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:32.338449 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b5e642fb-10a3-4d6e-ac88-eb6407624cf0 None None] MFA Rules not processed for user `dc3bac1dedef4f52a889f4835729c000`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:32.340642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b5e642fb-10a3-4d6e-ac88-eb6407624cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:32.460034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:32.469559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 984/1965] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:34.497531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=33,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:37.294678 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:37.297447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:37.446449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3d85d-e253-4a92-94ef-3a01dda653a8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:37.449951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d4ce3e02e5945ffbbde389398b146c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:37.450782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:37.451071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:37.451819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:37.541636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:37.543121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:37.552778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 982/1966] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:37.703583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4711c6f-bfae-41f0-99d7-f002b93c24f0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:37.704114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4711c6f-bfae-41f0-99d7-f002b93c24f0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:37.704114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4711c6f-bfae-41f0-99d7-f002b93c24f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:37.723859 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d4711c6f-bfae-41f0-99d7-f002b93c24f0 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:37.725362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d4711c6f-bfae-41f0-99d7-f002b93c24f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:37.789906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 985/1967] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:26:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:37.802473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:37.880839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478900c-b5ea-48dd-adab-1b24aa2edb76 service glance] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:37.886350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a703a1-a1f3-4174-b6cb-caadf7b4c0fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:37.887050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a703a1-a1f3-4174-b6cb-caadf7b4c0fb service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:37.887395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a703a1-a1f3-4174-b6cb-caadf7b4c0fb service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:37.887732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a703a1-a1f3-4174-b6cb-caadf7b4c0fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:37.898160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a703a1-a1f3-4174-b6cb-caadf7b4c0fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:37.899950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a703a1-a1f3-4174-b6cb-caadf7b4c0fb service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:37.910574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 983/1968] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:26:43.211101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbb7ea-3b34-4cc8-983d-17b4e94a01a2 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.211926 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbb7ea-3b34-4cc8-983d-17b4e94a01a2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.212404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbb7ea-3b34-4cc8-983d-17b4e94a01a2 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.219807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 986/1969] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:26:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:26:43.229928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d799c-969f-412b-ad25-d67247cf1775 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.230810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d799c-969f-412b-ad25-d67247cf1775 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.231464 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d799c-969f-412b-ad25-d67247cf1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.261030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-db1d799c-969f-412b-ad25-d67247cf1775 None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:43.262532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-db1d799c-969f-412b-ad25-d67247cf1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:43.343105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:43.417366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 984/1970] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:26:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:43.426326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbb7ea-3b34-4cc8-983d-17b4e94a01a2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:43.455695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbb7ea-3b34-4cc8-983d-17b4e94a01a2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:43.461061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ee881-cb40-40ac-a4d5-407808089c6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d4ce3e02e5945ffbbde389398b146c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:43.461702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ee881-cb40-40ac-a4d5-407808089c6d service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.461984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ee881-cb40-40ac-a4d5-407808089c6d service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.462256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ee881-cb40-40ac-a4d5-407808089c6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.478542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ee881-cb40-40ac-a4d5-407808089c6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:43.480391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ee881-cb40-40ac-a4d5-407808089c6d service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:43.493510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 987/1971] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:43.636980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffc6d5-a336-4060-8f4d-aadd9451de38 None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.637428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffc6d5-a336-4060-8f4d-aadd9451de38 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.637817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffc6d5-a336-4060-8f4d-aadd9451de38 None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.640990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 985/1972] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:26:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:26:43.647068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22532a5-4615-48cf-b419-e02e5a3b3148 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.647533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22532a5-4615-48cf-b419-e02e5a3b3148 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.647844 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22532a5-4615-48cf-b419-e02e5a3b3148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.673159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f22532a5-4615-48cf-b419-e02e5a3b3148 None None] MFA Rules not processed for user `b7cba7b6049e41488f5b2f7c61a0500a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:43.674385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f22532a5-4615-48cf-b419-e02e5a3b3148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:43.733667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 988/1973] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:26:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:43.749754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffc6d5-a336-4060-8f4d-aadd9451de38 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:43.765827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffc6d5-a336-4060-8f4d-aadd9451de38 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:43.769666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ea20f-e756-4b72-882a-34f5b5752799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:43.770359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea20f-e756-4b72-882a-34f5b5752799 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.770708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea20f-e756-4b72-882a-34f5b5752799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.771047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea20f-e756-4b72-882a-34f5b5752799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.779019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ea20f-e756-4b72-882a-34f5b5752799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:43.781019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ea20f-e756-4b72-882a-34f5b5752799 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:43.789984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 986/1974] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:43.882110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cea0d-d3ed-43de-b6a0-87e0506cd7b5 None None] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.882662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cea0d-d3ed-43de-b6a0-87e0506cd7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.882929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cea0d-d3ed-43de-b6a0-87e0506cd7b5 None None] PATH_INFO: `/` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.884597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 989/1975] 10.222.0.48 () {54 vars in 921 bytes} [Thu Oct 27 13:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:26:43.890983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1eec4-a85a-4783-b14c-58ec1edb428e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.892294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1eec4-a85a-4783-b14c-58ec1edb428e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.892294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1eec4-a85a-4783-b14c-58ec1edb428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.914338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-58e1eec4-a85a-4783-b14c-58ec1edb428e None None] MFA Rules not processed for user `d1d142bc3a9e4b3badefe713c7405329`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:43.915726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-58e1eec4-a85a-4783-b14c-58ec1edb428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:43.932660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 987/1976] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Oct 27 13:26:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:43.940208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cea0d-d3ed-43de-b6a0-87e0506cd7b5 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:43.953165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cea0d-d3ed-43de-b6a0-87e0506cd7b5 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:43.957481 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:43.958430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:43.958796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:43.961456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:43.973962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:43.975465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:43.992570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 990/1977] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:47.821369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fb734-b681-4c22-8c39-146b0279d6c7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:47.821870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fb734-b681-4c22-8c39-146b0279d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:47.821966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fb734-b681-4c22-8c39-146b0279d6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:47.841675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5d1fb734-b681-4c22-8c39-146b0279d6c7 None None] MFA Rules not processed for user `706ba757aacf4256b1416b5e3bbda714`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:47.842859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5d1fb734-b681-4c22-8c39-146b0279d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:47.851333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 988/1978] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:48.258679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:48.260187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:48.331009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b9057-e545-4745-a01a-1732229d87a6 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:48.336537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:48.337550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:48.337831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:48.338100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] PATH_INFO: `/v3/users/d1d126a141f540c3b1600cb195640b6c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:48.339256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d126a141f540c3b1600cb195640b6c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:48.341199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:48.380464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:48.398821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d126a141f540c3b1600cb195640b6c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:48.412756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d126a141f540c3b1600cb195640b6c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:48.413572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d126a141f540c3b1600cb195640b6c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:48.422143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d126a141f540c3b1600cb195640b6c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:48.433896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d126a141f540c3b1600cb195640b6c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:48.440733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 991/1979] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:26:48 2022] DELETE /identity/v3/users/d1d126a141f540c3b1600cb195640b6c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:26:48.445846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:48.446194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:48.446530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:48.466868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:48.468242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:48.577889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 989/1980] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:48.599782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:48.602731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:48.670348 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bee62-fd83-4d61-bd13-a7fcfe2d4004 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:48.674905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a4f91-0329-4fc4-a9fd-bf9bfc7def88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:48.675496 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a4f91-0329-4fc4-a9fd-bf9bfc7def88 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:48.675765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a4f91-0329-4fc4-a9fd-bf9bfc7def88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:48.676043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a4f91-0329-4fc4-a9fd-bf9bfc7def88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:48.686306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a4f91-0329-4fc4-a9fd-bf9bfc7def88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:48.687909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a4f91-0329-4fc4-a9fd-bf9bfc7def88 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:48.698857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 992/1981] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:49.040518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.041980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:49.056100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f747eab-91e3-4f79-bb31-73f07073897a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:49.064002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:49.064820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.065174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.067068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] PATH_INFO: `/v3/projects/412ef1dbb71f4d62956608eb8d404937` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.067890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] RBAC: Authorizing `identity:delete_project(project_id=412ef1dbb71f4d62956608eb8d404937)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:49.070134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:49.135241 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412ef1dbb71f4d62956608eb8d404937', 'request_id': 'req-0a7b29a8-f243-4789-aa68-712def12672b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:49.144288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412ef1dbb71f4d62956608eb8d404937', 'request_id': 'req-0a7b29a8-f243-4789-aa68-712def12672b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:49.150128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412ef1dbb71f4d62956608eb8d404937', 'request_id': 'req-0a7b29a8-f243-4789-aa68-712def12672b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:49.163404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] The token cache is being invalidate because project 412ef1dbb71f4d62956608eb8d404937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:26:49.163688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:49.164879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0a7b29a8-f243-4789-aa68-712def12672b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:26:49.166380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 990/1982] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:26:49 2022] DELETE /identity/v3/projects/412ef1dbb71f4d62956608eb8d404937 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:26:49.171130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7230f-a657-48f3-978d-bdad2d96d0dd None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.171415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7230f-a657-48f3-978d-bdad2d96d0dd None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.171687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7230f-a657-48f3-978d-bdad2d96d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.193689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cad7230f-a657-48f3-978d-bdad2d96d0dd None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:49.194912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cad7230f-a657-48f3-978d-bdad2d96d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:49.312792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 993/1983] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:49.314459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d5c4-3c09-4337-92d6-756dfb53a4ed None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.315068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d5c4-3c09-4337-92d6-756dfb53a4ed None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.315233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d5c4-3c09-4337-92d6-756dfb53a4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.353797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-6ea9d5c4-3c09-4337-92d6-756dfb53a4ed None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:49.355369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6ea9d5c4-3c09-4337-92d6-756dfb53a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:49.371271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 991/1984] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:49.376714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7230f-a657-48f3-978d-bdad2d96d0dd None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.396412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7230f-a657-48f3-978d-bdad2d96d0dd None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:49.403098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:49.403850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.404279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.404740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.405988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:26:49.406609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:49.411273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:49.432345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 994/1985] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:26:49.436166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9d5c4-3c09-4337-92d6-756dfb53a4ed None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.450927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9d5c4-3c09-4337-92d6-756dfb53a4ed None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:49.470613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:49.471303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.471962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.473038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.474112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:49.476289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:49.510886 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 992/1986] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:26:49 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:26:49.515037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.531450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60004508-fe9a-4417-b2e1-e809fbcb9b9e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:49.599802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:49.648031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:49.648756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.649313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.649643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.650312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:49.653814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:49.681166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 995/1987] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:26:49 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:26:49.686279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.698174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:49.704995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a69fe-1515-4bcf-acf8-cf03b077d5d3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:49.732355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:49.733390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.733856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.734528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.735792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:49.740893 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:49.768198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 993/1988] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:26:49.771548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.789082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e6ebf-46cd-4381-b91d-28394794035e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:49.796300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830caa88-f998-4dc4-bf82-95da240ac9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:49.797028 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830caa88-f998-4dc4-bf82-95da240ac9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.797176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830caa88-f998-4dc4-bf82-95da240ac9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.797464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830caa88-f998-4dc4-bf82-95da240ac9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.798088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830caa88-f998-4dc4-bf82-95da240ac9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:49.799688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830caa88-f998-4dc4-bf82-95da240ac9f3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:49.816067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 996/1989] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:26:49.817309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.828291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d12792-8755-412d-bd95-59db29b14a82 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:49.834817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:49.834817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.834817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.835122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] PATH_INFO: `/v3/projects/6ddba1e8bfeb4c8baf308e9b43bf1bee/users/c12f89d30c834651b659ceeaa8a0fd54/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.857913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ddba1e8bfeb4c8baf308e9b43bf1bee, user_id=c12f89d30c834651b659ceeaa8a0fd54, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:49.862193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:49.875880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 994/1990] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:26:49 2022] PUT /identity/v3/projects/6ddba1e8bfeb4c8baf308e9b43bf1bee/users/c12f89d30c834651b659ceeaa8a0fd54/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:26:49.884119 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ec546-de91-44ca-8ba0-c1d6d592103b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:49.884467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ec546-de91-44ca-8ba0-c1d6d592103b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:49.884771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ec546-de91-44ca-8ba0-c1d6d592103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:49.907772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6d8ec546-de91-44ca-8ba0-c1d6d592103b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:49.909504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6d8ec546-de91-44ca-8ba0-c1d6d592103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:49.978946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 997/1991] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:49.988970 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:49.993038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:50.064719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b7c67-ca06-4254-b25f-53a78e39d389 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:50.069813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:50.070345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:50.070625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:50.070898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:50.083717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:50.085669 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:50.096709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 995/1992] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:50.646913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a8f7e-d0cd-444f-93a3-aedb8af57827 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:50.648415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a8f7e-d0cd-444f-93a3-aedb8af57827 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:50.649285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a8f7e-d0cd-444f-93a3-aedb8af57827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:50.687009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f07a8f7e-d0cd-444f-93a3-aedb8af57827 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:50.689160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f07a8f7e-d0cd-444f-93a3-aedb8af57827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:50.842497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 998/1993] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:26:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:50.858287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:50.860025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:50.954613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6277aec-d280-4d3b-8e91-3deb45ed9496 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:50.954982 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6277aec-d280-4d3b-8e91-3deb45ed9496 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:50.955139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6277aec-d280-4d3b-8e91-3deb45ed9496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:51.000931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f6277aec-d280-4d3b-8e91-3deb45ed9496 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:51.002221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f6277aec-d280-4d3b-8e91-3deb45ed9496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:51.006005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a50deb-aaca-470d-843f-fea5680c4593 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:51.012849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd6308-d14c-4abb-b356-b8e1bd25d3d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:51.013972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd6308-d14c-4abb-b356-b8e1bd25d3d6 service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:51.014482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd6308-d14c-4abb-b356-b8e1bd25d3d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:51.015003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd6308-d14c-4abb-b356-b8e1bd25d3d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:51.022371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 999/1994] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:26:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:51.027620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd6308-d14c-4abb-b356-b8e1bd25d3d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:51.029864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd6308-d14c-4abb-b356-b8e1bd25d3d6 service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:51.032657 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6277aec-d280-4d3b-8e91-3deb45ed9496 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:51.049785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 996/1995] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 13:26:51.064456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6277aec-d280-4d3b-8e91-3deb45ed9496 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:51.073254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594f01d-75ac-4aff-ac9d-b627610c7375 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:51.074594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594f01d-75ac-4aff-ac9d-b627610c7375 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:51.075776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594f01d-75ac-4aff-ac9d-b627610c7375 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:51.076463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594f01d-75ac-4aff-ac9d-b627610c7375 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:51.089741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594f01d-75ac-4aff-ac9d-b627610c7375 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:51.091683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594f01d-75ac-4aff-ac9d-b627610c7375 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:51.105735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1000/1996] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:26:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 13:26:51.808481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:51.809551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:51.810080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:51.865553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] MFA Rules not processed for user `a385ba4ea6ee4a2fa76f0f12719427ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:51.869406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:52.114465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 997/1997] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:26:52.164077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292f592-6798-4d62-8b5f-7fb6b2c3f9a6 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:52.164704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292f592-6798-4d62-8b5f-7fb6b2c3f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:52.165230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292f592-6798-4d62-8b5f-7fb6b2c3f9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:52.311533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c292f592-6798-4d62-8b5f-7fb6b2c3f9a6 None None] MFA Rules not processed for user `c12f89d30c834651b659ceeaa8a0fd54`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:52.315202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c292f592-6798-4d62-8b5f-7fb6b2c3f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:52.419861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:52.482782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1001/1998] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:26:52.497168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:26:52.499266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:52.521173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:26:52.658948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d114b7-1dc1-41a4-aca4-db26fb774979 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:26:52.671872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c70858-7c14-4953-901c-5001b1de5c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:26:52.676972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70858-7c14-4953-901c-5001b1de5c5e service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:52.677582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70858-7c14-4953-901c-5001b1de5c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:52.678111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c70858-7c14-4953-901c-5001b1de5c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:52.697342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c70858-7c14-4953-901c-5001b1de5c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:26:52.704143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c70858-7c14-4953-901c-5001b1de5c5e service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:26:52.721083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 998/1999] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:26:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:26:54.739577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c681c7-a0dd-46dd-bbbb-963b1e5d2c45 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:26:54.740454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c681c7-a0dd-46dd-bbbb-963b1e5d2c45 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:26:54.740454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c681c7-a0dd-46dd-bbbb-963b1e5d2c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:26:54.764016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-20c681c7-a0dd-46dd-bbbb-963b1e5d2c45 None None] MFA Rules not processed for user `c12f89d30c834651b659ceeaa8a0fd54`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:26:54.765335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-20c681c7-a0dd-46dd-bbbb-963b1e5d2c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:26:54.772588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1002/2000] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:26:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:27:02.712316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=23 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:27:04.758668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=16 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:27:08.931237 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed756627-4cf9-4fc3-907c-27da5c592b4c None None] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:27:08.931237 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed756627-4cf9-4fc3-907c-27da5c592b4c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:27:08.932002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed756627-4cf9-4fc3-907c-27da5c592b4c None None] PATH_INFO: `/` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:27:08.934230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 999/2001] 10.222.0.48 () {54 vars in 961 bytes} [Thu Oct 27 13:27:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 27 13:27:08.942779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f58d0-2588-4e88-82ef-c5df8b8c74d3 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:27:08.943574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f58d0-2588-4e88-82ef-c5df8b8c74d3 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:27:08.953339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f58d0-2588-4e88-82ef-c5df8b8c74d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:27:09.034304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-170f58d0-2588-4e88-82ef-c5df8b8c74d3 None None] MFA Rules not processed for user `a578acf1e0cc4505a03c1b587b381b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:27:09.036064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-170f58d0-2588-4e88-82ef-c5df8b8c74d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:27:09.149041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1003/2002] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Oct 27 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:27:09.161077 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed756627-4cf9-4fc3-907c-27da5c592b4c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:27:09.190551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed756627-4cf9-4fc3-907c-27da5c592b4c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:27:09.196470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:27:09.197095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:27:09.197375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:27:09.197646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:27:09.199640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:27:09.215193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:27:09.217089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:27:09.232386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1000/2003] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:27:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:27:19.139834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=29 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:27:19.224811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:27:22.539793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:27:22.540539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:27:22.540898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:27:22.561774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:27:22.563063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:27:22.570224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1004/2004] 10.222.0.48 () {58 vars in 967 bytes} [Thu Oct 27 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:27:22.594957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:27:22.607884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed15e7-fa6f-4b54-87d2-c17499cccfaa service placement] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:27:22.614021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b421f-6fca-4e33-a5dc-391bd7c5b4ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:27:22.614242 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b421f-6fca-4e33-a5dc-391bd7c5b4ee service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:27:22.614412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b421f-6fca-4e33-a5dc-391bd7c5b4ee service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:27:22.614688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b421f-6fca-4e33-a5dc-391bd7c5b4ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:27:22.632158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b421f-6fca-4e33-a5dc-391bd7c5b4ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:27:22.633793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b421f-6fca-4e33-a5dc-391bd7c5b4ee service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:27:22.652982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1001/2005] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:27:30.209392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:27:30.210896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:27:30.223450 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64da1b-b318-446d-a5f7-2a4e45669a0f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:27:30.227498 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee5c77-9952-451d-8e94-aebf61360387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:27:30.228111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee5c77-9952-451d-8e94-aebf61360387 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:27:30.228531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee5c77-9952-451d-8e94-aebf61360387 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:27:30.228807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee5c77-9952-451d-8e94-aebf61360387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:27:30.230937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d1ee5c77-9952-451d-8e94-aebf61360387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:27:30.315329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee5c77-9952-451d-8e94-aebf61360387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:27:30.316807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee5c77-9952-451d-8e94-aebf61360387 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:27:30.326569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1005/2006] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:27:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:27:32.642483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:27:40.322052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=33 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:27:49.139793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b7d3b-642b-45ec-be76-bbc70fe08efb None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:27:49.140356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b7d3b-642b-45ec-be76-bbc70fe08efb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:27:49.140356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b7d3b-642b-45ec-be76-bbc70fe08efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:27:49.160405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-543b7d3b-642b-45ec-be76-bbc70fe08efb None None] MFA Rules not processed for user `97d696ea0031417d8a1a0b8471c26579`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:27:49.161613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-543b7d3b-642b-45ec-be76-bbc70fe08efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:27:49.216112 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1002/2007] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:27:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:27:59.209375 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=25 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:28:24.353965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64695b2-eac6-4b79-88ee-d2bc2691e076 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:28:24.354560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64695b2-eac6-4b79-88ee-d2bc2691e076 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:28:24.354831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64695b2-eac6-4b79-88ee-d2bc2691e076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:28:24.377471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e64695b2-eac6-4b79-88ee-d2bc2691e076 None None] MFA Rules not processed for user `c12f89d30c834651b659ceeaa8a0fd54`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:28:24.379835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e64695b2-eac6-4b79-88ee-d2bc2691e076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:28:24.386588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1006/2008] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:28:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:28:34.372034 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:28:41.643718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5088d-352b-4017-943e-a9eaa315c2a0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:28:41.644500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5088d-352b-4017-943e-a9eaa315c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:28:41.644500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5088d-352b-4017-943e-a9eaa315c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:28:41.668540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e0c5088d-352b-4017-943e-a9eaa315c2a0 None None] MFA Rules not processed for user `e77228bb97fd40dfa311088f28864637`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:28:41.669877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e0c5088d-352b-4017-943e-a9eaa315c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:28:41.756358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1003/2009] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:28:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:28:42.978457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509bcb9-fd0b-401e-b5b5-771ffebbc86a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:28:42.978972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509bcb9-fd0b-401e-b5b5-771ffebbc86a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:28:42.979301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509bcb9-fd0b-401e-b5b5-771ffebbc86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:28:43.012206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b509bcb9-fd0b-401e-b5b5-771ffebbc86a None None] MFA Rules not processed for user `8f0bb483c548429295c02558520c6898`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:28:43.012206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b509bcb9-fd0b-401e-b5b5-771ffebbc86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:28:43.137865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1007/2010] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:28:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:28:50.206423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5088d-352b-4017-943e-a9eaa315c2a0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:28:50.230898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5088d-352b-4017-943e-a9eaa315c2a0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:28:50.236989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:28:50.237920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:28:50.238918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:28:50.239553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:28:50.243719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:28:50.336468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:28:50.337959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:28:50.350300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1004/2011] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:28:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:28:51.472680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509bcb9-fd0b-401e-b5b5-771ffebbc86a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:28:51.482446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509bcb9-fd0b-401e-b5b5-771ffebbc86a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:28:51.485904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:28:51.486495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:28:51.486762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:28:51.487023 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:28:51.488791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:28:51.502302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:28:51.503963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:28:51.514870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1008/2012] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:28:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:00.344784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=59 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:01.510292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=31 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:03.485266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:03.486748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:03.504047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff342cc-0cd3-4700-8e82-48895425f5b0 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:03.510985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:03.511926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:03.512522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:03.512855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:03.515033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:03.528817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:03.530502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:03.541371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1005/2013] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:04.977185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:04.979891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:05.001257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01e974-8c6e-42a9-8629-3f6c509ebd78 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:05.023230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baec511-0ae7-4419-a063-bbc799a34ad2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:05.024321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baec511-0ae7-4419-a063-bbc799a34ad2 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:05.024736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baec511-0ae7-4419-a063-bbc799a34ad2 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:05.025154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baec511-0ae7-4419-a063-bbc799a34ad2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:05.028251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7baec511-0ae7-4419-a063-bbc799a34ad2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:05.048878 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baec511-0ae7-4419-a063-bbc799a34ad2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:05.050689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baec511-0ae7-4419-a063-bbc799a34ad2 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:05.061891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1009/2014] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:29:05.307054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:05.323266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680f02b-caaa-48de-aad2-63f154a6e5aa service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:05.331322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:05.331971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:05.332619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:05.332899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:05.336003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:05.358880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:05.361860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:05.375358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1006/2015] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:06.727375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d81f2-4214-46dd-88de-8d49b4a4a445 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:06.727836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d81f2-4214-46dd-88de-8d49b4a4a445 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:06.728052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d81f2-4214-46dd-88de-8d49b4a4a445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:06.769205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0f6d81f2-4214-46dd-88de-8d49b4a4a445 None None] MFA Rules not processed for user `a385ba4ea6ee4a2fa76f0f12719427ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:06.770382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0f6d81f2-4214-46dd-88de-8d49b4a4a445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:06.787725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1010/2016] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:07.341919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:07.351821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69811a0-dd97-4527-94d7-0f76694f3c30 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:07.355526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:07.356134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:07.356456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:07.356736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:07.358444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:07.370082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:07.371512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:07.380251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1007/2017] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:08.169335 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d81f2-4214-46dd-88de-8d49b4a4a445 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:08.183923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d81f2-4214-46dd-88de-8d49b4a4a445 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:08.189057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:08.190214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:08.190663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:08.191918 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:08.195678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:08.216508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:08.216508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:08.229101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1011/2018] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:29:16.523074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:16.532416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6db7cb-62a9-4f3b-9e13-3ac3fda2c871 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:16.537396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:16.537918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:16.538201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:16.538476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:16.548916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:16.561159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:16.562590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:16.572266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1008/2019] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:18.225274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=15 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:23.106600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:23.118682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab21716-5305-448e-a634-adcce815f1f1 service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:23.124181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:23.124817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:23.125093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:23.125362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:23.127082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:23.141929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:23.143502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:23.155027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1012/2020] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:24.838838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:24.847860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dfbe2-5834-4da1-bc1d-3a530fadc6f8 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:24.851495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:24.852092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:24.852393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:24.852665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:24.854358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:24.866962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:24.868718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:24.879100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1009/2021] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:28.362899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:28.382236 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83d2ba-122c-405b-81dd-125cd94c3a5b service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:28.394330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:28.395489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:28.395942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:28.396443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:28.400508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:28.429410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:28.433908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:28.456605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1013/2022] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:28.460032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:28.462068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:28.485333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3569e60-37a9-4bf3-8338-bbf5a7ea1100 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:28.494500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:28.495165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:28.501335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:28.501651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] PATH_INFO: `/v3/users/97d696ea0031417d8a1a0b8471c26579` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:28.503387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d696ea0031417d8a1a0b8471c26579)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:28.505428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:28.561621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d696ea0031417d8a1a0b8471c26579'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.577715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d696ea0031417d8a1a0b8471c26579'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.578450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d696ea0031417d8a1a0b8471c26579'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.587547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d696ea0031417d8a1a0b8471c26579'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.601260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ff5bb2fe-ad58-409d-9a62-ae05571d3139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d696ea0031417d8a1a0b8471c26579'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.607708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1010/2023] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:28 2022] DELETE /identity/v3/users/97d696ea0031417d8a1a0b8471c26579 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:28.781298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:28.793548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca8f2d-bc23-4ea0-a033-4337b02b2422 service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:28.852832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:28.853394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:28.853668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:28.853966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] PATH_INFO: `/v3/projects/a58f7c57ef7f4a54999d16d765d80cc2` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:28.855071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58f7c57ef7f4a54999d16d765d80cc2)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:28.856998 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:28.904293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58f7c57ef7f4a54999d16d765d80cc2', 'request_id': 'req-d6dd5bed-b325-41d0-8648-d163e9001548'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.911722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58f7c57ef7f4a54999d16d765d80cc2', 'request_id': 'req-d6dd5bed-b325-41d0-8648-d163e9001548'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.917199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58f7c57ef7f4a54999d16d765d80cc2', 'request_id': 'req-d6dd5bed-b325-41d0-8648-d163e9001548'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.928702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] The token cache is being invalidate because project a58f7c57ef7f4a54999d16d765d80cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:28.929008 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.929666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-d6dd5bed-b325-41d0-8648-d163e9001548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:28.931228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1014/2024] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:28 2022] DELETE /identity/v3/projects/a58f7c57ef7f4a54999d16d765d80cc2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:28.936309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce6e9-800c-4281-a2fe-efa9b7cb899a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:28.936600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce6e9-800c-4281-a2fe-efa9b7cb899a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:28.936878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce6e9-800c-4281-a2fe-efa9b7cb899a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:28.957109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-727ce6e9-800c-4281-a2fe-efa9b7cb899a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:28.958935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-727ce6e9-800c-4281-a2fe-efa9b7cb899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:29.017355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1011/2025] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:29.025696 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2322c25-f3d9-4fdc-8389-d1e12f6e1c0f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.025984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2322c25-f3d9-4fdc-8389-d1e12f6e1c0f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.026264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2322c25-f3d9-4fdc-8389-d1e12f6e1c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.045728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a2322c25-f3d9-4fdc-8389-d1e12f6e1c0f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:29.046933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a2322c25-f3d9-4fdc-8389-d1e12f6e1c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:29.054933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1015/2026] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:29.059336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce6e9-800c-4281-a2fe-efa9b7cb899a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.069522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce6e9-800c-4281-a2fe-efa9b7cb899a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.074119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.074721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.075000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.075272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.075945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:29:29.076251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.077872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.079402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:29.091963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1012/2027] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:29:29.096448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2322c25-f3d9-4fdc-8389-d1e12f6e1c0f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.105702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2322c25-f3d9-4fdc-8389-d1e12f6e1c0f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.110361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.110969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.111249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.111518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.112148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.114086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.143164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1016/2028] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:29 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:29.148129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.158600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59538b-7038-4ee3-bd76-84618e37c75c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.229297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.230530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.230530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.230530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.231095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.233040 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.259022 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1013/2029] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:29 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:29.262909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.274046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e43db5-a93e-4f65-9570-2ca1315c0f61 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.279316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.280018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.280385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.280712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.281414 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.283074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.300907 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1017/2030] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:29.305508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.315683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90051fdc-4bd7-465b-bac3-c5f101f0ef83 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.321265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96c133-952e-40fd-bc6a-47b932372283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.321878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c133-952e-40fd-bc6a-47b932372283 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.322167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c133-952e-40fd-bc6a-47b932372283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.322440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96c133-952e-40fd-bc6a-47b932372283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.323080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96c133-952e-40fd-bc6a-47b932372283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.324712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96c133-952e-40fd-bc6a-47b932372283 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.340879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1014/2031] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:29.344011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.355777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe61462-93af-4319-963d-c3a64db25e12 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.361594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.362396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.362744 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.363306 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] PATH_INFO: `/v3/projects/8bbd62fe6e20414aaa784fe64befa90b/users/63374897c7cd435c86ae3370301de5ab/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.381927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bbd62fe6e20414aaa784fe64befa90b, user_id=63374897c7cd435c86ae3370301de5ab, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.384854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.400601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:29.412539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1018/2032] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:29 2022] PUT /identity/v3/projects/8bbd62fe6e20414aaa784fe64befa90b/users/63374897c7cd435c86ae3370301de5ab/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:29.418570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ada4e-c84f-4796-ac03-c78d35eba959 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.418940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ada4e-c84f-4796-ac03-c78d35eba959 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.419282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ada4e-c84f-4796-ac03-c78d35eba959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.455113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ca9ada4e-c84f-4796-ac03-c78d35eba959 None None] MFA Rules not processed for user `63374897c7cd435c86ae3370301de5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:29.456561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ca9ada4e-c84f-4796-ac03-c78d35eba959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:29.540794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1015/2033] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:29.545359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.620185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7a237-5f9a-4920-bced-a4f19c1de245 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.625012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.625717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.626006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.626277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.626934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.629340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.668385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1019/2034] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:29 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:29.668981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ada4e-c84f-4796-ac03-c78d35eba959 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.681360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ada4e-c84f-4796-ac03-c78d35eba959 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.747608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.748251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.749265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.749265 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.749845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.751911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.763103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:29.780030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1016/2035] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:29 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:29.784165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.804705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6ee1-8bf4-4dd5-828d-5a110660ec42 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.810284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.811682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.811770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.812074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.812641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.815208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.835399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.836312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1020/2036] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:29.847388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530da2-50ae-4e8f-b34c-75ef7d01b881 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.853516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.854196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.854569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.854836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] PATH_INFO: `/v3/projects/85c58dfe37ea408487e2ebd114b1cadd/users/e740f17772d746bcbc9f087cf4259ff9/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.870585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c58dfe37ea408487e2ebd114b1cadd, user_id=e740f17772d746bcbc9f087cf4259ff9, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.873180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.888467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1017/2037] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:29 2022] PUT /identity/v3/projects/85c58dfe37ea408487e2ebd114b1cadd/users/e740f17772d746bcbc9f087cf4259ff9/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:29.892598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.903015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e84df-1929-4b53-98ee-a7d3f92fbbc4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:29.969240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51ede-4cb1-4ea8-b886-bbfc6b8433bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:29.969910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ede-4cb1-4ea8-b886-bbfc6b8433bd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:29.970187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ede-4cb1-4ea8-b886-bbfc6b8433bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:29.970448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51ede-4cb1-4ea8-b886-bbfc6b8433bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:29.971222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51ede-4cb1-4ea8-b886-bbfc6b8433bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:29.972880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51ede-4cb1-4ea8-b886-bbfc6b8433bd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:29.984147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1021/2038] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:29:29.988438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:29.999768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fea02-9c4c-427e-8519-57b9a3b87455 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:30.009313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:30.010067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:30.010347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:30.010619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] PATH_INFO: `/v3/projects/85c58dfe37ea408487e2ebd114b1cadd/users/e740f17772d746bcbc9f087cf4259ff9/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:30.012600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c58dfe37ea408487e2ebd114b1cadd, user_id=e740f17772d746bcbc9f087cf4259ff9, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:30.015245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:30.026861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1018/2039] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:29 2022] PUT /identity/v3/projects/85c58dfe37ea408487e2ebd114b1cadd/users/e740f17772d746bcbc9f087cf4259ff9/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:30.037259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:30.037677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:30.038053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:30.070555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] MFA Rules not processed for user `e740f17772d746bcbc9f087cf4259ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:30.072063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:30.094165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:30.095957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:30.155646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:30.167909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1022/2040] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:29:30.178687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0247900-42c7-4eee-b858-f9e42518dfb3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:30.180401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:30.181970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:30.183466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:30.184107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:30.184418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:30.184686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] PATH_INFO: `/v3/users/c12f89d30c834651b659ceeaa8a0fd54` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:30.185882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c12f89d30c834651b659ceeaa8a0fd54)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:30.187829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:30.245074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c12f89d30c834651b659ceeaa8a0fd54'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.250300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9c5db-52ba-4836-9718-194681749b0c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:30.255018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:30.255808 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:30.256084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:30.256383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:30.261060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c12f89d30c834651b659ceeaa8a0fd54'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.261785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c12f89d30c834651b659ceeaa8a0fd54'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.266991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:30.268608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:30.269985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c12f89d30c834651b659ceeaa8a0fd54'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.280031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1023/2041] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:29:30.280959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c12f89d30c834651b659ceeaa8a0fd54'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.287708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1019/2042] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:30 2022] DELETE /identity/v3/users/c12f89d30c834651b659ceeaa8a0fd54 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:29:30.490467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:30.510751 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c897a-2381-4746-9876-b064cbe6139a service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:30.600330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:30.600945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:30.601219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:30.601806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] PATH_INFO: `/v3/projects/6ddba1e8bfeb4c8baf308e9b43bf1bee` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:30.603146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ddba1e8bfeb4c8baf308e9b43bf1bee)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:30.605108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:30.665996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ddba1e8bfeb4c8baf308e9b43bf1bee', 'request_id': 'req-23b6da2e-1dbe-42bc-8264-39bc597bc886'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.674726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ddba1e8bfeb4c8baf308e9b43bf1bee', 'request_id': 'req-23b6da2e-1dbe-42bc-8264-39bc597bc886'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.681555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ddba1e8bfeb4c8baf308e9b43bf1bee', 'request_id': 'req-23b6da2e-1dbe-42bc-8264-39bc597bc886'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.695607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] The token cache is being invalidate because project 6ddba1e8bfeb4c8baf308e9b43bf1bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:30.695938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.696832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-23b6da2e-1dbe-42bc-8264-39bc597bc886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.698726 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1024/2043] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:30 2022] DELETE /identity/v3/projects/6ddba1e8bfeb4c8baf308e9b43bf1bee => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:30.779709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:30.781764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:30.798352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3f460-d1af-4a09-b6fa-20103d345cd0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:30.846908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:30.882372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:30.883227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:30.883606 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:30.883978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] PATH_INFO: `/v3/users/63374897c7cd435c86ae3370301de5ab` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:30.885362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] RBAC: Authorizing `identity:delete_user(user_id=63374897c7cd435c86ae3370301de5ab)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:30.887482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:30.945512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63374897c7cd435c86ae3370301de5ab'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.959291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63374897c7cd435c86ae3370301de5ab'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.960016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63374897c7cd435c86ae3370301de5ab'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.967431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63374897c7cd435c86ae3370301de5ab'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.976463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63374897c7cd435c86ae3370301de5ab'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:30.982768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1020/2044] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:30 2022] DELETE /identity/v3/users/63374897c7cd435c86ae3370301de5ab => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:30.987218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:30.987508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:30.987784 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:31.008807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:31.010102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:31.037560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:31.084386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1025/2045] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:31.092991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:31.094673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:31.164658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c6db-dce8-4455-b055-eea7cd61730b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:31.168574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:31.169217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:31.169497 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:31.169794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:31.180981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:31.182666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:31.194046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1021/2046] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:29:31.634854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:31.637006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:31.663507 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a709055-66ff-4986-8901-7504ee9e4122 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:31.668271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:31.668915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:31.669321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:31.669600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] PATH_INFO: `/v3/projects/8bbd62fe6e20414aaa784fe64befa90b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:31.670768 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bbd62fe6e20414aaa784fe64befa90b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:31.672745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:31.726665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bbd62fe6e20414aaa784fe64befa90b', 'request_id': 'req-351868a1-4b84-4536-bdfb-ce74a69a687d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.736413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bbd62fe6e20414aaa784fe64befa90b', 'request_id': 'req-351868a1-4b84-4536-bdfb-ce74a69a687d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.747007 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bbd62fe6e20414aaa784fe64befa90b', 'request_id': 'req-351868a1-4b84-4536-bdfb-ce74a69a687d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.762537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] The token cache is being invalidate because project 8bbd62fe6e20414aaa784fe64befa90b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:31.763208 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.764203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.766574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1026/2047] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:31 2022] DELETE /identity/v3/projects/8bbd62fe6e20414aaa784fe64befa90b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:31.770615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:31.780879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9452b-9504-40e8-bb5d-e1d7d5a6a49d service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:31.846448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:31.857542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:31.858604 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:31.859094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:31.859513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] PATH_INFO: `/v3/users/e740f17772d746bcbc9f087cf4259ff9` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:31.861167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e740f17772d746bcbc9f087cf4259ff9)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:31.864001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:31.918584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e740f17772d746bcbc9f087cf4259ff9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.930540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e740f17772d746bcbc9f087cf4259ff9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.931327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e740f17772d746bcbc9f087cf4259ff9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.939774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e740f17772d746bcbc9f087cf4259ff9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.949447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dd867fe5-e47b-4255-b8a9-7513c16dce87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e740f17772d746bcbc9f087cf4259ff9'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:31.955811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1022/2048] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:31 2022] DELETE /identity/v3/users/e740f17772d746bcbc9f087cf4259ff9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:29:32.249670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:32.251220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:32.265086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351868a1-4b84-4536-bdfb-ce74a69a687d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:32.322454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:32.323082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.323352 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.323618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] PATH_INFO: `/v3/projects/85c58dfe37ea408487e2ebd114b1cadd` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.324741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c58dfe37ea408487e2ebd114b1cadd)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:32.326615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:32.373891 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c58dfe37ea408487e2ebd114b1cadd', 'request_id': 'req-65a1ded6-7cab-4c41-9058-d2ebff987d31'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:32.377786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:32.383415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c58dfe37ea408487e2ebd114b1cadd', 'request_id': 'req-65a1ded6-7cab-4c41-9058-d2ebff987d31'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:32.390130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c58dfe37ea408487e2ebd114b1cadd', 'request_id': 'req-65a1ded6-7cab-4c41-9058-d2ebff987d31'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:32.401503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] The token cache is being invalidate because project 85c58dfe37ea408487e2ebd114b1cadd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:32.401793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:32.402452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-65a1ded6-7cab-4c41-9058-d2ebff987d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:32.403950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1027/2049] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:32 2022] DELETE /identity/v3/projects/85c58dfe37ea408487e2ebd114b1cadd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:32.408918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff9523-f681-48ec-9453-16f5bb18bd0c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.409205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff9523-f681-48ec-9453-16f5bb18bd0c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.409483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff9523-f681-48ec-9453-16f5bb18bd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.429790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-a6ff9523-f681-48ec-9453-16f5bb18bd0c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:32.430973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a6ff9523-f681-48ec-9453-16f5bb18bd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:32.488918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1023/2050] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:32.497248 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b6c1-f59c-4473-8170-db83c9d66e3b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.497494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b6c1-f59c-4473-8170-db83c9d66e3b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.497786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b6c1-f59c-4473-8170-db83c9d66e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.519319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3470b6c1-f59c-4473-8170-db83c9d66e3b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:32.520673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3470b6c1-f59c-4473-8170-db83c9d66e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:32.529736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1028/2051] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:32.534779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff9523-f681-48ec-9453-16f5bb18bd0c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:32.547108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff9523-f681-48ec-9453-16f5bb18bd0c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:32.553740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:32.554644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.555000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.555328 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.556122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:29:32.556683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:32.559350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:32.577064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1024/2052] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:29:32.581815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b6c1-f59c-4473-8170-db83c9d66e3b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:32.595397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b6c1-f59c-4473-8170-db83c9d66e3b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:32.602735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:32.603363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.603646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.603920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.604898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:32.607321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:32.644046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1029/2053] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:32 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:32.650065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:32.661558 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982055-cc31-4fdd-942a-5dd0822babd7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:32.729296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:32.729905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.730175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.730437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.731073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:32.733207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:32.742964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:32.763921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1025/2054] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:32 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:32.768153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:32.778718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3135c-a0a4-4cc3-850e-2ca990c639be admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:32.783562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:32.784155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.784462 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.784734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.785999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:32.787593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:32.798280 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1030/2055] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:32.803044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:32.813902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7549dc-e75e-42e1-afdc-6af1c987f980 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:32.818596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc1691-24f0-4053-91ad-0aa262d7c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:32.819193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc1691-24f0-4053-91ad-0aa262d7c656 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.819467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc1691-24f0-4053-91ad-0aa262d7c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.819732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc1691-24f0-4053-91ad-0aa262d7c656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.820456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc1691-24f0-4053-91ad-0aa262d7c656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:32.822044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc1691-24f0-4053-91ad-0aa262d7c656 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:32.833055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1026/2056] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:29:32.840492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:32.853984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db257a3-386a-4143-a956-70fe5e408fb6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:32.858565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:32.859212 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.859483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.859751 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] PATH_INFO: `/v3/projects/149bc7c3b34f4a2e86dfbf06d528e831/users/770b5186ef2b41c0b1a0240a34ae5828/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.875778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=149bc7c3b34f4a2e86dfbf06d528e831, user_id=770b5186ef2b41c0b1a0240a34ae5828, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:32.878742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:32.896736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1031/2057] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:32 2022] PUT /identity/v3/projects/149bc7c3b34f4a2e86dfbf06d528e831/users/770b5186ef2b41c0b1a0240a34ae5828/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:29:32.915342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542cf42-fa6e-4f56-b498-926a1e5df848 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:32.915716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542cf42-fa6e-4f56-b498-926a1e5df848 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:32.916093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542cf42-fa6e-4f56-b498-926a1e5df848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:32.948252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-7542cf42-fa6e-4f56-b498-926a1e5df848 None None] MFA Rules not processed for user `770b5186ef2b41c0b1a0240a34ae5828`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:32.949492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7542cf42-fa6e-4f56-b498-926a1e5df848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:33.014345 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1027/2058] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:33.018776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.084056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef675105-119a-4c34-a0b9-a611ccba1fcf admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.089158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.089778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.090053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.090322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.090965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.092903 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.122114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1032/2059] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:33 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:33.126302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542cf42-fa6e-4f56-b498-926a1e5df848 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.137504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542cf42-fa6e-4f56-b498-926a1e5df848 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.194971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.195583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.195864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.196136 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.196798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.198701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.223507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1028/2060] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:33 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:33.226262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.236688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53262fd-09fd-423a-ba4a-ee1d42ef1ee1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.242776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.243761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.244250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.244758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.245871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.248866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.268311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1033/2061] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:33.273445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.283752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d1e37-75c0-4ddf-9378-e43c2d250b1a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.289071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.289732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.290013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.290897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] PATH_INFO: `/v3/projects/f4b3590ececf42fba5a74b9f1a5a4c7b/users/4735792f1109424f939b666f1a44f4ce/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.306894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b3590ececf42fba5a74b9f1a5a4c7b, user_id=4735792f1109424f939b666f1a44f4ce, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.309564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.335266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1029/2062] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:33 2022] PUT /identity/v3/projects/f4b3590ececf42fba5a74b9f1a5a4c7b/users/4735792f1109424f939b666f1a44f4ce/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:33.339947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.350945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120caab-ce8b-4e10-be89-88ad87d68444 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.402533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:33.415407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac972a6-fd9a-4cea-b099-768ec6077eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.416406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac972a6-fd9a-4cea-b099-768ec6077eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.416822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac972a6-fd9a-4cea-b099-768ec6077eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.417127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac972a6-fd9a-4cea-b099-768ec6077eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.417879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac972a6-fd9a-4cea-b099-768ec6077eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.419992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac972a6-fd9a-4cea-b099-768ec6077eb2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.431677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1034/2063] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:29:33.435863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.446472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675393f-55c9-41b7-bb49-1aa7f0c7fbf3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.451052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.451729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.452002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.452311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] PATH_INFO: `/v3/projects/f4b3590ececf42fba5a74b9f1a5a4c7b/users/4735792f1109424f939b666f1a44f4ce/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.454218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b3590ececf42fba5a74b9f1a5a4c7b, user_id=4735792f1109424f939b666f1a44f4ce, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.456881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.468035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1030/2064] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:33 2022] PUT /identity/v3/projects/f4b3590ececf42fba5a74b9f1a5a4c7b/users/4735792f1109424f939b666f1a44f4ce/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:33.479562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.479839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.480138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.511251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] MFA Rules not processed for user `4735792f1109424f939b666f1a44f4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:33.512493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:33.585708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1035/2065] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:33.596796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.599220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:33.626376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:33.680251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba1351-a5f8-4536-a4ac-ae0f7c492d84 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.686064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.687088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.687535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.687982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.704034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.706612 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.725235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1031/2066] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:29:33.775576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:33.777033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:33.790894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346b467-cb07-45a9-90ed-6aa7c0d2b320 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:33.794788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c32249-5422-468e-bc44-ccae4c550769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:33.796006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c32249-5422-468e-bc44-ccae4c550769 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:33.796006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c32249-5422-468e-bc44-ccae4c550769 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:33.796006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c32249-5422-468e-bc44-ccae4c550769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:33.862379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c32249-5422-468e-bc44-ccae4c550769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:33.863840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c32249-5422-468e-bc44-ccae4c550769 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:33.873537 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1036/2067] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:33 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:37.321069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:37.395131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a8bb4-2aee-4537-89ec-cce8c41d6ee9 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:37.399790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:37.400456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:37.400731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:37.400996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] PATH_INFO: `/v3/users/770b5186ef2b41c0b1a0240a34ae5828` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:37.402121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770b5186ef2b41c0b1a0240a34ae5828)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:37.404032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:37.458679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770b5186ef2b41c0b1a0240a34ae5828'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:37.471513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770b5186ef2b41c0b1a0240a34ae5828'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:37.472334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770b5186ef2b41c0b1a0240a34ae5828'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:37.480291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770b5186ef2b41c0b1a0240a34ae5828'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:37.488141 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770b5186ef2b41c0b1a0240a34ae5828'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:37.494315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1032/2068] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:37 2022] DELETE /identity/v3/users/770b5186ef2b41c0b1a0240a34ae5828 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:37.498858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:37.499813 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:37.499813 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:37.524572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:37.526447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:37.589272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1037/2069] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:37.605695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:37.607617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:37.661547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:37.676554 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8259cca-5223-4c25-9c39-1f06aa7d06a9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:37.680342 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:37.681012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:37.681293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:37.681560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:37.691745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:37.693450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:37.705122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1033/2070] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:38.026837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:38.028854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:38.043663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1025c-6004-408c-9cf6-0f68fb2ed81b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:38.049847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:38.051142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:38.052550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:38.052550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] PATH_INFO: `/v3/projects/149bc7c3b34f4a2e86dfbf06d528e831` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:38.054561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149bc7c3b34f4a2e86dfbf06d528e831)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:38.057368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:38.072545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:38.142406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149bc7c3b34f4a2e86dfbf06d528e831', 'request_id': 'req-78bad344-fbf6-40c0-84eb-ba4e46d04538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.150187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149bc7c3b34f4a2e86dfbf06d528e831', 'request_id': 'req-78bad344-fbf6-40c0-84eb-ba4e46d04538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.164662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149bc7c3b34f4a2e86dfbf06d528e831', 'request_id': 'req-78bad344-fbf6-40c0-84eb-ba4e46d04538'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.173274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] The token cache is being invalidate because project 149bc7c3b34f4a2e86dfbf06d528e831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:38.173692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.174691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.177562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1038/2071] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:38 2022] DELETE /identity/v3/projects/149bc7c3b34f4a2e86dfbf06d528e831 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:38.182533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:38.194398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496dce0-db9f-4737-a3ef-d8fe1134716e service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:38.259639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:38.260435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:38.260761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:38.261086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] PATH_INFO: `/v3/users/4735792f1109424f939b666f1a44f4ce` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:38.262466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4735792f1109424f939b666f1a44f4ce)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:38.264854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:38.319536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4735792f1109424f939b666f1a44f4ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.332649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4735792f1109424f939b666f1a44f4ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.333694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4735792f1109424f939b666f1a44f4ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.341734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4735792f1109424f939b666f1a44f4ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.357630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0ecf81a9-6dda-44af-a772-b91dcc962ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4735792f1109424f939b666f1a44f4ce'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.368761 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1034/2072] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:38 2022] DELETE /identity/v3/users/4735792f1109424f939b666f1a44f4ce => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:38.671320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:38.672913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:38.686928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad344-fbf6-40c0-84eb-ba4e46d04538 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:38.750112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:38.750695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:38.750991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:38.751287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] PATH_INFO: `/v3/projects/f4b3590ececf42fba5a74b9f1a5a4c7b` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:38.752801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b3590ececf42fba5a74b9f1a5a4c7b)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:38.755380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:38.806685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b3590ececf42fba5a74b9f1a5a4c7b', 'request_id': 'req-b5ada568-a065-48c1-8922-e0381fda17d5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.817957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b3590ececf42fba5a74b9f1a5a4c7b', 'request_id': 'req-b5ada568-a065-48c1-8922-e0381fda17d5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.824207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b3590ececf42fba5a74b9f1a5a4c7b', 'request_id': 'req-b5ada568-a065-48c1-8922-e0381fda17d5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.837266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] The token cache is being invalidate because project f4b3590ececf42fba5a74b9f1a5a4c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:38.837562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.838195 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b5ada568-a065-48c1-8922-e0381fda17d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:38.839668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1039/2073] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:38 2022] DELETE /identity/v3/projects/f4b3590ececf42fba5a74b9f1a5a4c7b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:38.845398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50330936-c7cd-4508-bef3-7a6a3b9eed3d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:38.845768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50330936-c7cd-4508-bef3-7a6a3b9eed3d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:38.846048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50330936-c7cd-4508-bef3-7a6a3b9eed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:38.867514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-50330936-c7cd-4508-bef3-7a6a3b9eed3d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:38.868768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-50330936-c7cd-4508-bef3-7a6a3b9eed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:38.938022 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1035/2074] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:29:38.947534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1ee05-03d5-4543-b283-720041cbb0fb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:38.947817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1ee05-03d5-4543-b283-720041cbb0fb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:38.948090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1ee05-03d5-4543-b283-720041cbb0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:38.969484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3ba1ee05-03d5-4543-b283-720041cbb0fb None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:38.970460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3ba1ee05-03d5-4543-b283-720041cbb0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:38.979019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1040/2075] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:38.983721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50330936-c7cd-4508-bef3-7a6a3b9eed3d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.000801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50330936-c7cd-4508-bef3-7a6a3b9eed3d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.008712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.009649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.010089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.010542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.011610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:29:39.012083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.014869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.017107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:39.037196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1036/2076] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:29:39.038847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1ee05-03d5-4543-b283-720041cbb0fb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.048467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1ee05-03d5-4543-b283-720041cbb0fb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.054202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.054825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.055112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.055407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.056134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.058134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.086753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:39.100222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1041/2077] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:39 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:39.101456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.112045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233bb50-6611-4d4a-bd56-ba9a45a40b35 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.180119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.180767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.181043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.181339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.182095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.184492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.206994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1037/2078] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:39 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:39.211605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.226152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a40b5-17bb-45c4-872f-1d28dda8949e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.233659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.234865 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.235177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.235625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.236699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.239424 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.250066 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1042/2079] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:39.255180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.266397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdce43-28a9-47dd-bca8-6032b3137ea1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.272204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991b850-43c4-4a15-87e7-3e06742c9d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.272860 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991b850-43c4-4a15-87e7-3e06742c9d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.273146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991b850-43c4-4a15-87e7-3e06742c9d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.273417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991b850-43c4-4a15-87e7-3e06742c9d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.274067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991b850-43c4-4a15-87e7-3e06742c9d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.275683 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991b850-43c4-4a15-87e7-3e06742c9d9d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.290195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1038/2080] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:39.294499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.308365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba60bb2-bf01-4067-a17f-1dd5b81e5699 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.313814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.314475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.314752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.315016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] PATH_INFO: `/v3/projects/4c76707839aa4d769e46fa47d99cc834/users/e8843e07e1564466bd16505d3f527471/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.331986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c76707839aa4d769e46fa47d99cc834, user_id=e8843e07e1564466bd16505d3f527471, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.334698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.351532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1043/2081] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:39 2022] PUT /identity/v3/projects/4c76707839aa4d769e46fa47d99cc834/users/e8843e07e1564466bd16505d3f527471/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:39.362030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8822-ea84-4c9d-8594-b9c369698eab None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.362611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8822-ea84-4c9d-8594-b9c369698eab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.362611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8822-ea84-4c9d-8594-b9c369698eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.394803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-703c8822-ea84-4c9d-8594-b9c369698eab None None] MFA Rules not processed for user `e8843e07e1564466bd16505d3f527471`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:39.396174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-703c8822-ea84-4c9d-8594-b9c369698eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:39.467801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1039/2082] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:39.473084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.545663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813b4fd-6d22-4871-860b-6bebc161c611 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.550217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.551399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.551677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.551945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.552620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.554514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.583958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1044/2083] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:39 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:39.589227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c8822-ea84-4c9d-8594-b9c369698eab None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.602109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c8822-ea84-4c9d-8594-b9c369698eab None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.667511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.668125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.668432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.668701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.669336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.671386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.680999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:39.694785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1040/2084] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:39 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:39.699036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.710117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba218b0-607c-431c-9766-3fdc3211f63c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.714868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.715465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.715740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.716006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.716668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.718345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.730525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1045/2085] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:39.734906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.747306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36ec1-c1e1-4552-b4c5-64f6cbab7288 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.754169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.754834 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.755103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.755369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] PATH_INFO: `/v3/projects/c9dbc6c1b16d4d999a2e2cc41830eb3e/users/4e339f143ec643efb2a8ca9ead97923d/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.774520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9dbc6c1b16d4d999a2e2cc41830eb3e, user_id=4e339f143ec643efb2a8ca9ead97923d, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.777208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.798476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1041/2086] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:39 2022] PUT /identity/v3/projects/c9dbc6c1b16d4d999a2e2cc41830eb3e/users/4e339f143ec643efb2a8ca9ead97923d/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:39.807920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.818646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaa74e-6585-4804-9a39-52212641ff71 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.881336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b934f9-019d-46f5-932d-ad7438606ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.881954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b934f9-019d-46f5-932d-ad7438606ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.882265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b934f9-019d-46f5-932d-ad7438606ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.882535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b934f9-019d-46f5-932d-ad7438606ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.883203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b934f9-019d-46f5-932d-ad7438606ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.884835 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b934f9-019d-46f5-932d-ad7438606ed1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.896720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1046/2087] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:39.905532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:39.918140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5c6ef-acb7-4477-a244-c6cd9c93a808 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:39.922843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:39.923526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.923795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.924067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] PATH_INFO: `/v3/projects/c9dbc6c1b16d4d999a2e2cc41830eb3e/users/4e339f143ec643efb2a8ca9ead97923d/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.926058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9dbc6c1b16d4d999a2e2cc41830eb3e, user_id=4e339f143ec643efb2a8ca9ead97923d, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:39.928750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:39.941862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1042/2088] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:39 2022] PUT /identity/v3/projects/c9dbc6c1b16d4d999a2e2cc41830eb3e/users/4e339f143ec643efb2a8ca9ead97923d/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:39.955625 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee25a1-25c2-48dd-8d8c-c90a983fba66 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:39.955904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee25a1-25c2-48dd-8d8c-c90a983fba66 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:39.956170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee25a1-25c2-48dd-8d8c-c90a983fba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:39.990095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9dee25a1-25c2-48dd-8d8c-c90a983fba66 None None] MFA Rules not processed for user `4e339f143ec643efb2a8ca9ead97923d`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:39.991345 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9dee25a1-25c2-48dd-8d8c-c90a983fba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:40.017378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:40.077271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1047/2089] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:40.089758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:40.092161 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:40.159023 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd04d-3347-413c-8df2-c70ce7fa7620 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:40.163715 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:40.164424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:40.164744 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:40.165043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:40.175938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:40.177527 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:40.188879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1043/2090] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:40.310578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee25a1-25c2-48dd-8d8c-c90a983fba66 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:40.320816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee25a1-25c2-48dd-8d8c-c90a983fba66 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:40.324354 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb6d23-7817-4bfb-96b4-86165041a6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:40.324951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb6d23-7817-4bfb-96b4-86165041a6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:40.325247 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb6d23-7817-4bfb-96b4-86165041a6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:40.325520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb6d23-7817-4bfb-96b4-86165041a6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:40.406562 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb6d23-7817-4bfb-96b4-86165041a6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:40.407988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb6d23-7817-4bfb-96b4-86165041a6c2 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:40.417420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1048/2091] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:43.088039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:43.164126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e87fc-9724-4648-a980-ce666af688c2 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:43.168776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:43.169413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:43.169734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:43.169954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] PATH_INFO: `/v3/users/e8843e07e1564466bd16505d3f527471` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:43.171091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8843e07e1564466bd16505d3f527471)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:43.173058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:43.232137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8843e07e1564466bd16505d3f527471'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.241998 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:43.249403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8843e07e1564466bd16505d3f527471'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.250258 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8843e07e1564466bd16505d3f527471'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.260420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8843e07e1564466bd16505d3f527471'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.268618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8843e07e1564466bd16505d3f527471'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.275331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1044/2092] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:43 2022] DELETE /identity/v3/users/e8843e07e1564466bd16505d3f527471 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:43.280068 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:43.280576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:43.280717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:43.302489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:43.303713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:43.368542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1049/2093] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:43.378569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:43.380005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:43.441206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535cf3a-3ef9-4435-9ab6-4818da44c842 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:43.444841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:43.445437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:43.445708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:43.445974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:43.455800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:43.457789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:43.468958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1045/2094] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:43.773235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:43.774850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:43.787919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f72832-2fce-4b28-becd-ef6e81fb817f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:43.792968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:43.793616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:43.793897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:43.794255 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] PATH_INFO: `/v3/projects/4c76707839aa4d769e46fa47d99cc834` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:43.795411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c76707839aa4d769e46fa47d99cc834)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:43.797519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:43.857572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c76707839aa4d769e46fa47d99cc834', 'request_id': 'req-82de77d1-d592-4189-88ef-c7e6df87ee7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.860319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:43.866408 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c76707839aa4d769e46fa47d99cc834', 'request_id': 'req-82de77d1-d592-4189-88ef-c7e6df87ee7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.873346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c76707839aa4d769e46fa47d99cc834', 'request_id': 'req-82de77d1-d592-4189-88ef-c7e6df87ee7a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.886332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] The token cache is being invalidate because project 4c76707839aa4d769e46fa47d99cc834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:43.886612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.887340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:43.889054 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1050/2095] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:43 2022] DELETE /identity/v3/projects/4c76707839aa4d769e46fa47d99cc834 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:43.895387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:43.906996 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf245a-0e8f-4edb-b58c-5638476866a5 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:43.968945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:43.969736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:43.970078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:43.970370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] PATH_INFO: `/v3/users/4e339f143ec643efb2a8ca9ead97923d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:43.971640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e339f143ec643efb2a8ca9ead97923d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:43.973690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:44.026484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e339f143ec643efb2a8ca9ead97923d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.043082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e339f143ec643efb2a8ca9ead97923d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.043357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e339f143ec643efb2a8ca9ead97923d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.054836 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e339f143ec643efb2a8ca9ead97923d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.062963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8f7ba63f-5e26-46fc-986c-208b405f6685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e339f143ec643efb2a8ca9ead97923d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.065194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:44.076017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1046/2096] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:43 2022] DELETE /identity/v3/users/4e339f143ec643efb2a8ca9ead97923d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:44.388582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:44.390881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:44.414532 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de77d1-d592-4189-88ef-c7e6df87ee7a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:44.485966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:44.486551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:44.486827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:44.487097 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] PATH_INFO: `/v3/projects/c9dbc6c1b16d4d999a2e2cc41830eb3e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:44.488799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9dbc6c1b16d4d999a2e2cc41830eb3e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:44.490737 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:44.554205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9dbc6c1b16d4d999a2e2cc41830eb3e', 'request_id': 'req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.564993 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9dbc6c1b16d4d999a2e2cc41830eb3e', 'request_id': 'req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.576689 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9dbc6c1b16d4d999a2e2cc41830eb3e', 'request_id': 'req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.599324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] The token cache is being invalidate because project c9dbc6c1b16d4d999a2e2cc41830eb3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:44.599603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.600342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-20faf469-dc5e-4e8d-b5db-9a3ff98854f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:44.601812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1051/2097] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:44 2022] DELETE /identity/v3/projects/c9dbc6c1b16d4d999a2e2cc41830eb3e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:44.607819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acc42b-7810-4230-bf09-0f25805bdd50 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:44.608794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acc42b-7810-4230-bf09-0f25805bdd50 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:44.609084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acc42b-7810-4230-bf09-0f25805bdd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:44.635409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-70acc42b-7810-4230-bf09-0f25805bdd50 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:44.637285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-70acc42b-7810-4230-bf09-0f25805bdd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:44.710259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1047/2098] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:44.721039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87474fb2-8755-4eb7-85a6-da54dea875d9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:44.721753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87474fb2-8755-4eb7-85a6-da54dea875d9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:44.722869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87474fb2-8755-4eb7-85a6-da54dea875d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:44.747810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-87474fb2-8755-4eb7-85a6-da54dea875d9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:44.750360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-87474fb2-8755-4eb7-85a6-da54dea875d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:44.759899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1052/2099] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:44.765035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acc42b-7810-4230-bf09-0f25805bdd50 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:44.779496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acc42b-7810-4230-bf09-0f25805bdd50 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:44.787364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:44.788358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:44.788819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:44.789198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:44.790156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:29:44.790599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:44.792415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:44.808714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1048/2100] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:29:44.814456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87474fb2-8755-4eb7-85a6-da54dea875d9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:44.824326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87474fb2-8755-4eb7-85a6-da54dea875d9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:44.829241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:44.829837 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:44.830107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:44.830372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:44.831004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:44.832939 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:44.873182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1053/2101] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:44 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:44.878391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:44.903075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7bba4-716f-4be9-827c-43c6fc67b39d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:44.983435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:44.984051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:44.984503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:44.984668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:44.985426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:44.987340 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.012119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1049/2102] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:44 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:45.020373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.031411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5143dfd-7b0b-4eea-9d57-4da3c89c7d8f admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.037006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.037610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.037888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.038159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.038796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.040451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.050833 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1054/2103] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:45.055096 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.065072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a939d-dfe0-41bd-8934-66847663c7bd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.070616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf00d57-e39b-4a49-8342-c3e27442250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.071246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf00d57-e39b-4a49-8342-c3e27442250b admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.071529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf00d57-e39b-4a49-8342-c3e27442250b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.071792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf00d57-e39b-4a49-8342-c3e27442250b admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.072471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf00d57-e39b-4a49-8342-c3e27442250b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.074261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf00d57-e39b-4a49-8342-c3e27442250b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.090073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1050/2104] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:45.091400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.102132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312f872-fee5-49d2-9077-d37c23948d4d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.107191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.107929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.108266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.108572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] PATH_INFO: `/v3/projects/8391b61043364b4484b386d584d3b0eb/users/6fda711c505241d19f7b6a143cc83d24/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.124691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8391b61043364b4484b386d584d3b0eb, user_id=6fda711c505241d19f7b6a143cc83d24, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.128814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.144805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1055/2105] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:45 2022] PUT /identity/v3/projects/8391b61043364b4484b386d584d3b0eb/users/6fda711c505241d19f7b6a143cc83d24/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:45.155381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65016-3cfa-42d0-ae5a-88d17bc80e0d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.155784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65016-3cfa-42d0-ae5a-88d17bc80e0d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.155954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65016-3cfa-42d0-ae5a-88d17bc80e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.188165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-32a65016-3cfa-42d0-ae5a-88d17bc80e0d None None] MFA Rules not processed for user `6fda711c505241d19f7b6a143cc83d24`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:45.189419 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-32a65016-3cfa-42d0-ae5a-88d17bc80e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:45.255607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:45.268204 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1051/2106] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:45.273972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.284095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:45.351539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27c911-7b51-489c-90a3-a0dbc9dbf2a7 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.356388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.357001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.357275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.357543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.358194 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.360120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.388731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1056/2107] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:45 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:45.394940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65016-3cfa-42d0-ae5a-88d17bc80e0d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.406050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65016-3cfa-42d0-ae5a-88d17bc80e0d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.475583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.476260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.476528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.476835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.477545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.479666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.503721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1052/2108] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:45 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:45.507818 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.519281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90a64-5649-43fa-9fdf-969f1ef7825a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.525259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.526118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.526599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.527079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.527984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.530972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.553619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1057/2109] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:45.558071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.570823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35814eee-118b-4d41-87f7-fddb45a20693 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.577052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4c26c-36b2-447a-ba76-b279ddcfa011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.577672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4c26c-36b2-447a-ba76-b279ddcfa011 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.577953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4c26c-36b2-447a-ba76-b279ddcfa011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.578226 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4c26c-36b2-447a-ba76-b279ddcfa011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.578903 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4c26c-36b2-447a-ba76-b279ddcfa011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.580552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4c26c-36b2-447a-ba76-b279ddcfa011 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.595163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1053/2110] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:29:45.600359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.613489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f239f0a-4d5e-49d6-9b90-73d212f6468e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.620553 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.621283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.621687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.622076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] PATH_INFO: `/v3/projects/7fd5e4a901794643950cae522655579c/users/993bbe38e3914eaf983e874b5857ca40/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.642309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd5e4a901794643950cae522655579c, user_id=993bbe38e3914eaf983e874b5857ca40, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.645923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.658615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1058/2111] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:45 2022] PUT /identity/v3/projects/7fd5e4a901794643950cae522655579c/users/993bbe38e3914eaf983e874b5857ca40/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:45.670121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e947a-10ac-410d-9b9b-a053f81b1a04 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.670622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e947a-10ac-410d-9b9b-a053f81b1a04 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.670622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e947a-10ac-410d-9b9b-a053f81b1a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.702611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8c8e947a-10ac-410d-9b9b-a053f81b1a04 None None] MFA Rules not processed for user `993bbe38e3914eaf983e874b5857ca40`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:45.704223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8c8e947a-10ac-410d-9b9b-a053f81b1a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:45.779449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1054/2112] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:45.785615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.850855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e1068-d67e-4f62-895f-b1ff54d9dfd1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.855756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.856429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.856706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.856979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.857709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.859771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.889746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1059/2113] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:45 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:45.894202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e947a-10ac-410d-9b9b-a053f81b1a04 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:45.905125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e947a-10ac-410d-9b9b-a053f81b1a04 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:45.968503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:45.969196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:45.969529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:45.969854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:45.970652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:45.972933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:45.983084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:45.997929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1055/2114] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:45 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:29:46.003649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:46.015478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe5efd-001c-441c-8c4a-ce87a1ddc253 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:46.027272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:46.027977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:46.028278 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:46.028560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:46.029218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:46.030933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:46.042257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1060/2115] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:46.047353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:46.063879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88991a87-ab43-4011-b1cb-41ce928bf6ac admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:46.071843 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:46.072922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:46.073352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:46.073768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] PATH_INFO: `/v3/projects/714821843fad415f9e347e64cdb8cacf/users/3559b091dec540b4a75c15e2ff05187b/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:46.095011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714821843fad415f9e347e64cdb8cacf, user_id=3559b091dec540b4a75c15e2ff05187b, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:46.098482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:46.120799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1056/2116] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:46 2022] PUT /identity/v3/projects/714821843fad415f9e347e64cdb8cacf/users/3559b091dec540b4a75c15e2ff05187b/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:46.125015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:46.138949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529dd5b-3e5d-4061-8c37-e37b16956b6c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:46.182219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:46.215541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8655059-952f-4074-8132-070b77d22a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:46.216153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8655059-952f-4074-8132-070b77d22a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:46.216476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8655059-952f-4074-8132-070b77d22a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:46.216739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8655059-952f-4074-8132-070b77d22a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:46.217471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8655059-952f-4074-8132-070b77d22a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:46.219086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8655059-952f-4074-8132-070b77d22a21 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:46.235311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1061/2117] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:46.240770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:46.251383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8e3a6-23b0-4e73-bd6f-c4e4b1cab7f7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:46.263739 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:46.264529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:46.264814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:46.265125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] PATH_INFO: `/v3/projects/714821843fad415f9e347e64cdb8cacf/users/3559b091dec540b4a75c15e2ff05187b/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:46.267274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=714821843fad415f9e347e64cdb8cacf, user_id=3559b091dec540b4a75c15e2ff05187b, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:46.269972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:46.281273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1057/2118] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:46 2022] PUT /identity/v3/projects/714821843fad415f9e347e64cdb8cacf/users/3559b091dec540b4a75c15e2ff05187b/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:46.292336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:46.292677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:46.292888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:46.323106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] MFA Rules not processed for user `3559b091dec540b4a75c15e2ff05187b`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:46.324379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:46.416183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1062/2119] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:29:46.429963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:46.431670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:46.494312 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5de965-308f-48ed-ac1c-9bc911bbd5fe admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:46.498019 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:46.498653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:46.498936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:46.499202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:46.508591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:46.510007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:46.522546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1058/2120] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:46.577698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:46.579678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:46.593685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70adb-bac2-47e7-a894-46ac6c0a907c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:46.598148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca2169-b226-4680-98da-c2384a9c78a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:46.598776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca2169-b226-4680-98da-c2384a9c78a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:46.599045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca2169-b226-4680-98da-c2384a9c78a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:46.599311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca2169-b226-4680-98da-c2384a9c78a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:46.670684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca2169-b226-4680-98da-c2384a9c78a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:46.672136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca2169-b226-4680-98da-c2384a9c78a2 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:46.682743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1063/2121] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:49.624811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:49.686559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c89633-d06f-48ad-9ba6-2b12695a63e9 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:49.691707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:49.692412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:49.692702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:49.692972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] PATH_INFO: `/v3/users/6fda711c505241d19f7b6a143cc83d24` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:49.694095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fda711c505241d19f7b6a143cc83d24)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:49.696010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:49.741659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fda711c505241d19f7b6a143cc83d24'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:49.749469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:49.755287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fda711c505241d19f7b6a143cc83d24'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:49.756029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fda711c505241d19f7b6a143cc83d24'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:49.763590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fda711c505241d19f7b6a143cc83d24'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:49.770742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fda711c505241d19f7b6a143cc83d24'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:49.776716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1059/2122] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:49 2022] DELETE /identity/v3/users/6fda711c505241d19f7b6a143cc83d24 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:49.781898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:49.782392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:49.782489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:49.801460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:49.802674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:49.848830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:49.864102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1064/2123] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:49.873757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:49.875172 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:49.934115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a4909-846d-4295-965e-573f2e9f7ef3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:49.937892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:49.938362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:49.938639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:49.938910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:49.948346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:49.949756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:49.959185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1060/2124] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:50.222370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:50.223766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:50.235549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee64a44-dafc-4df3-86c0-6b6cc79f07fe None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:50.240680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:50.241312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:50.241601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:50.241856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] PATH_INFO: `/v3/projects/8391b61043364b4484b386d584d3b0eb` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:50.242961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8391b61043364b4484b386d584d3b0eb)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:50.244885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:50.291640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8391b61043364b4484b386d584d3b0eb', 'request_id': 'req-13284f9a-a37c-4031-8a61-694c2d82f0cc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.298827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8391b61043364b4484b386d584d3b0eb', 'request_id': 'req-13284f9a-a37c-4031-8a61-694c2d82f0cc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.305226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8391b61043364b4484b386d584d3b0eb', 'request_id': 'req-13284f9a-a37c-4031-8a61-694c2d82f0cc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.316992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] The token cache is being invalidate because project 8391b61043364b4484b386d584d3b0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:50.317277 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.317944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.319416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1065/2125] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:50 2022] DELETE /identity/v3/projects/8391b61043364b4484b386d584d3b0eb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:50.323547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:50.333273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43434e-318d-4779-9950-7035e0cf0bdc service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:50.390817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:50.391413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:50.391655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:50.391919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] PATH_INFO: `/v3/users/993bbe38e3914eaf983e874b5857ca40` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:50.393058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=993bbe38e3914eaf983e874b5857ca40)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:50.395063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:50.442098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993bbe38e3914eaf983e874b5857ca40'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.453614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993bbe38e3914eaf983e874b5857ca40'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.454266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993bbe38e3914eaf983e874b5857ca40'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.461151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993bbe38e3914eaf983e874b5857ca40'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.468238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993bbe38e3914eaf983e874b5857ca40'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.469736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:50.476293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1061/2126] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:50 2022] DELETE /identity/v3/users/993bbe38e3914eaf983e874b5857ca40 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:50.751358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:50.753597 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:50.767806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13284f9a-a37c-4031-8a61-694c2d82f0cc admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:50.825398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:50.826082 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:50.826291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:50.826558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] PATH_INFO: `/v3/projects/7fd5e4a901794643950cae522655579c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:50.827646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd5e4a901794643950cae522655579c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:50.829558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:50.874439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd5e4a901794643950cae522655579c', 'request_id': 'req-0806567c-45b7-40c1-ae5d-6ed353b6b354'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.882796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd5e4a901794643950cae522655579c', 'request_id': 'req-0806567c-45b7-40c1-ae5d-6ed353b6b354'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.889678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd5e4a901794643950cae522655579c', 'request_id': 'req-0806567c-45b7-40c1-ae5d-6ed353b6b354'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.902944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] The token cache is being invalidate because project 7fd5e4a901794643950cae522655579c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:50.903219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.903923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:50.905449 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1066/2127] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:50 2022] DELETE /identity/v3/projects/7fd5e4a901794643950cae522655579c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:50.909898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:50.911421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:50.928759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934e495-ff58-4092-bf6e-f2f8acb56c5e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:50.999800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:51.000555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.000831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.001097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] PATH_INFO: `/v3/users/3559b091dec540b4a75c15e2ff05187b` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.002520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3559b091dec540b4a75c15e2ff05187b)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:51.005263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:51.055083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3559b091dec540b4a75c15e2ff05187b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.071941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3559b091dec540b4a75c15e2ff05187b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.072635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3559b091dec540b4a75c15e2ff05187b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.085643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3559b091dec540b4a75c15e2ff05187b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.098213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6e59c6a9-4016-41a1-bb0f-b031889c7de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3559b091dec540b4a75c15e2ff05187b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.109952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1062/2128] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:29:50 2022] DELETE /identity/v3/users/3559b091dec540b4a75c15e2ff05187b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:51.418923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:51.420535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:51.429627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:51.433747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806567c-45b7-40c1-ae5d-6ed353b6b354 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:51.524519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:51.525502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.525942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.526368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] PATH_INFO: `/v3/projects/714821843fad415f9e347e64cdb8cacf` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.528157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=714821843fad415f9e347e64cdb8cacf)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:51.531756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:51.600371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714821843fad415f9e347e64cdb8cacf', 'request_id': 'req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.609167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714821843fad415f9e347e64cdb8cacf', 'request_id': 'req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.615006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714821843fad415f9e347e64cdb8cacf', 'request_id': 'req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.630226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] The token cache is being invalidate because project 714821843fad415f9e347e64cdb8cacf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:29:51.630512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.631163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ce4d037f-0ef5-4c14-9b2c-05550e006fd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:29:51.632800 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1067/2129] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:29:51 2022] DELETE /identity/v3/projects/714821843fad415f9e347e64cdb8cacf => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:51.638643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1702-64b3-4de9-8173-b0dadab86e70 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.638928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1702-64b3-4de9-8173-b0dadab86e70 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.639230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1702-64b3-4de9-8173-b0dadab86e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.659556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-629a1702-64b3-4de9-8173-b0dadab86e70 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:51.660795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-629a1702-64b3-4de9-8173-b0dadab86e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:51.722060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1063/2130] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:29:51.730967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceccb32-a334-42ef-90b6-d10fd75fa383 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.731249 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceccb32-a334-42ef-90b6-d10fd75fa383 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.731524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceccb32-a334-42ef-90b6-d10fd75fa383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.752373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1ceccb32-a334-42ef-90b6-d10fd75fa383 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:51.754202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1ceccb32-a334-42ef-90b6-d10fd75fa383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:51.771238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1068/2131] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:51.775511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a1702-64b3-4de9-8173-b0dadab86e70 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:51.785978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a1702-64b3-4de9-8173-b0dadab86e70 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:51.790734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:51.791334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.791608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.791871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.792543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:29:51.792839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:51.794449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:51.807089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1064/2132] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:29:51.811705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceccb32-a334-42ef-90b6-d10fd75fa383 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:51.821750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceccb32-a334-42ef-90b6-d10fd75fa383 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:51.826373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:51.826961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.827239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.827503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.828136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:51.830056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:51.858072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1069/2133] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:29:51 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:51.862749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:51.869098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:51.873215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc50708-3145-4de6-8c77-6bfe0404a827 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:51.933818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:51.934485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.934677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.934942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.935666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:51.937584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:51.959483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1065/2134] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:29:51 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:29:51.963920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:51.974849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765f367-d31d-4c34-9754-02c04d629691 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:51.979403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:51.980014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:51.980323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:51.980588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:51.981238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:51.982827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:51.993582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1070/2135] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:51.997645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:52.008810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9186e10-6365-4db8-811a-dbfd31d6f9b8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:52.013469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef7f41-f102-4204-8bae-47a2cbe1d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:52.014047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef7f41-f102-4204-8bae-47a2cbe1d32d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:52.014325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef7f41-f102-4204-8bae-47a2cbe1d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:52.014595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef7f41-f102-4204-8bae-47a2cbe1d32d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:52.015231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef7f41-f102-4204-8bae-47a2cbe1d32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:52.016855 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef7f41-f102-4204-8bae-47a2cbe1d32d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:52.027816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1066/2136] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:29:52.031957 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:52.041956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcb48a-ebac-47e3-8fff-e8ccce303cd3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:52.046577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:52.047233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:52.047501 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:52.047770 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] PATH_INFO: `/v3/projects/f68979c23550439f9e5938b08b4ce348/users/31e9ee130951455a91039ce530583ebc/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:52.062932 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68979c23550439f9e5938b08b4ce348, user_id=31e9ee130951455a91039ce530583ebc, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:52.066216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:52.079486 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1071/2137] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:29:52 2022] PUT /identity/v3/projects/f68979c23550439f9e5938b08b4ce348/users/31e9ee130951455a91039ce530583ebc/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:29:52.147329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:52.147607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:52.147876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:52.177006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] MFA Rules not processed for user `31e9ee130951455a91039ce530583ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:29:52.178211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:52.243817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1067/2138] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:29:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:29:52.255468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:52.257723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:52.292627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:52.317764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b5990-a6ee-40a7-9d30-3129fe3f3fdd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:52.321344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc1c3-7597-4ddc-bcb0-a1f20e1aae8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d4ce3e02e5945ffbbde389398b146c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:52.321951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc1c3-7597-4ddc-bcb0-a1f20e1aae8d service glance] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:52.322223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc1c3-7597-4ddc-bcb0-a1f20e1aae8d service glance] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:52.322489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc1c3-7597-4ddc-bcb0-a1f20e1aae8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:52.331936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc1c3-7597-4ddc-bcb0-a1f20e1aae8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:52.333406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc1c3-7597-4ddc-bcb0-a1f20e1aae8d service glance] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:52.342269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1072/2139] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:29:54.260276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:29:54.263144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:54.401661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17fe3c-29f3-440a-8a13-ad8b7f2b202b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:29:54.409493 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:29:54.410515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:29:54.411005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:29:54.411484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:29:54.416259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:29:54.430967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:29:54.433186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:29:54.439399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:29:54.450647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1068/2140] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:29:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:01.892304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:01.892846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:01.892992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:01.915260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:01.916552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:01.990211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1073/2141] 10.222.0.48 () {58 vars in 967 bytes} [Thu Oct 27 13:30:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:02.061679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:02.063406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:02.151097 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfd456-af0d-4ccc-97d2-dd295102e6b3 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:02.156815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:02.157434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:02.157706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:02.157972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:02.168069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:02.169698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:02.179852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1069/2142] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:30:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 13:30:10.497728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:10.499593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:10.578731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917687b-5657-490d-86f6-416b3e70d41a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:10.584326 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:10.584981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:10.585257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:10.585526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] PATH_INFO: `/v3/users/706ba757aacf4256b1416b5e3bbda714` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:10.586743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=706ba757aacf4256b1416b5e3bbda714)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:10.588715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:10.650089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706ba757aacf4256b1416b5e3bbda714'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.663082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:10.664821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706ba757aacf4256b1416b5e3bbda714'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.665540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706ba757aacf4256b1416b5e3bbda714'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.673975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706ba757aacf4256b1416b5e3bbda714'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.677285 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead57818-8de3-4ac5-88a1-9733f24382d0 service swift] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:10.682239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-3668e8a2-a7b4-47ff-b53e-821f9c7598d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706ba757aacf4256b1416b5e3bbda714'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.682458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:10.683162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:10.683449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:10.683719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] PATH_INFO: `/v3/users/31e9ee130951455a91039ce530583ebc` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:10.684290 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:10.685480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31e9ee130951455a91039ce530583ebc)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:10.687588 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:10.690396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1074/2143] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:10 2022] DELETE /identity/v3/users/706ba757aacf4256b1416b5e3bbda714 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:10.746015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31e9ee130951455a91039ce530583ebc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.764459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31e9ee130951455a91039ce530583ebc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.765211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31e9ee130951455a91039ce530583ebc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.778515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31e9ee130951455a91039ce530583ebc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.789809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31e9ee130951455a91039ce530583ebc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:10.797503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1070/2144] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:10 2022] DELETE /identity/v3/users/31e9ee130951455a91039ce530583ebc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:30:10.802643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:10.802934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:10.803220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:10.825670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:10.826924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:10.901246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1075/2145] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:10.911961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:10.913452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:10.917492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:10.919094 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:10.931508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe2143-bb2a-43ab-93cb-bdf17f8d1232 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:10.937327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:10.938026 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:10.938308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:10.938576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] PATH_INFO: `/v3/projects/52552d69672a4677bd52c847cceec247` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:10.940384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52552d69672a4677bd52c847cceec247)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:10.942327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:10.992207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcace9-a94a-4559-97d0-3909cbba70e3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:10.998171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52552d69672a4677bd52c847cceec247', 'request_id': 'req-a5f907d6-eea1-42f4-977c-4d3324568b20'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.005732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:11.008305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52552d69672a4677bd52c847cceec247', 'request_id': 'req-a5f907d6-eea1-42f4-977c-4d3324568b20'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.015368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52552d69672a4677bd52c847cceec247', 'request_id': 'req-a5f907d6-eea1-42f4-977c-4d3324568b20'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.029656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] The token cache is being invalidate because project 52552d69672a4677bd52c847cceec247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:11.030086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.030783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.032339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1076/2146] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:10 2022] DELETE /identity/v3/projects/52552d69672a4677bd52c847cceec247 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:30:11.036331 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:11.037643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:11.053990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:11.054607 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:11.054899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:11.055166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:11.056945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:11.106149 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f907d6-eea1-42f4-977c-4d3324568b20 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:11.111552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:11.112466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:11.112892 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:11.113499 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] PATH_INFO: `/v3/users/dc3bac1dedef4f52a889f4835729c000` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:11.115470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3bac1dedef4f52a889f4835729c000)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:11.118069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:11.133881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:11.135727 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:11.147416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:11.149418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1071/2147] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:11.182160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3bac1dedef4f52a889f4835729c000'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.202342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3bac1dedef4f52a889f4835729c000'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.203134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3bac1dedef4f52a889f4835729c000'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.215238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3bac1dedef4f52a889f4835729c000'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.226801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-058edb42-89b7-47a3-86df-5b9b425dc168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3bac1dedef4f52a889f4835729c000'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:11.239645 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1077/2148] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:11 2022] DELETE /identity/v3/users/dc3bac1dedef4f52a889f4835729c000 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:11.318802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:11.322335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:11.476795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62424e49-c4a8-46c8-b71c-2129d12dd68a service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:11.622148 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:11.636654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:11.637472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:11.637818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:11.638162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:11.640186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:11.658041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:11.659997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:11.688165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1072/2149] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:11.803402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:11.804391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:11.804911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:11.830687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:11.831945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:11.908718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1078/2150] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:30:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:11.921274 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:11.922791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:11.973381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:11.974941 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:12.038370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c6558-a48f-4bf4-8124-74f26333db33 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.050747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.052993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.054562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.055415 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.073468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087499b8-8a6d-4d3a-a18e-3b9592ff3e2c None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.073781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.075693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.078713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.080029 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.080847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.081195 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] PATH_INFO: `/v3/projects/9daea5edf5364f639b31d351f960bcc1` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.082504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] RBAC: Authorizing `identity:delete_project(project_id=9daea5edf5364f639b31d351f960bcc1)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.084654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.086268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1073/2151] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:30:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:30:12.087026 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.088395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:12.104945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b67317-4525-4891-9609-be5fa050467c service swift] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.110474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.112024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.112267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.112566 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] PATH_INFO: `/v3/projects/f68979c23550439f9e5938b08b4ce348` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.114663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68979c23550439f9e5938b08b4ce348)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.116323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.197658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:12.202318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9daea5edf5364f639b31d351f960bcc1', 'request_id': 'req-64ca1275-1187-40b2-b8ba-e1c1c8d468af'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.223871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9daea5edf5364f639b31d351f960bcc1', 'request_id': 'req-64ca1275-1187-40b2-b8ba-e1c1c8d468af'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.240817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9daea5edf5364f639b31d351f960bcc1', 'request_id': 'req-64ca1275-1187-40b2-b8ba-e1c1c8d468af'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.245564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68979c23550439f9e5938b08b4ce348', 'request_id': 'req-362d4fbc-5b1d-4f3f-9644-43f47f77370d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.268006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] The token cache is being invalidate because project 9daea5edf5364f639b31d351f960bcc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:12.269708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68979c23550439f9e5938b08b4ce348', 'request_id': 'req-362d4fbc-5b1d-4f3f-9644-43f47f77370d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.276253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.278551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-64ca1275-1187-40b2-b8ba-e1c1c8d468af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.279832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68979c23550439f9e5938b08b4ce348', 'request_id': 'req-362d4fbc-5b1d-4f3f-9644-43f47f77370d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.286775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1079/2152] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:11 2022] DELETE /identity/v3/projects/9daea5edf5364f639b31d351f960bcc1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:30:12.292661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a97ca-d7ce-4fed-bb26-f15f7edf6e8a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.293198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a97ca-d7ce-4fed-bb26-f15f7edf6e8a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.293665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a97ca-d7ce-4fed-bb26-f15f7edf6e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.304689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] The token cache is being invalidate because project f68979c23550439f9e5938b08b4ce348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:12.305303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.306416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-362d4fbc-5b1d-4f3f-9644-43f47f77370d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:12.314068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1074/2153] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:12 2022] DELETE /identity/v3/projects/f68979c23550439f9e5938b08b4ce348 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:12.315841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c63f-105a-4c1b-84a1-68a31f97ef5b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.316348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c63f-105a-4c1b-84a1-68a31f97ef5b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.316776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c63f-105a-4c1b-84a1-68a31f97ef5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.333612 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a95a97ca-d7ce-4fed-bb26-f15f7edf6e8a None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:12.337417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a95a97ca-d7ce-4fed-bb26-f15f7edf6e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:12.343391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-eb06c63f-105a-4c1b-84a1-68a31f97ef5b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:12.345208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-eb06c63f-105a-4c1b-84a1-68a31f97ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:12.446238 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1080/2154] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:12.449244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71f818-1848-46b3-a7a5-7836163a2ac4 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.449534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71f818-1848-46b3-a7a5-7836163a2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.449803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71f818-1848-46b3-a7a5-7836163a2ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.476031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ef71f818-1848-46b3-a7a5-7836163a2ac4 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:12.477767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ef71f818-1848-46b3-a7a5-7836163a2ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:12.486156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1081/2155] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:12.490602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71f818-1848-46b3-a7a5-7836163a2ac4 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.501003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71f818-1848-46b3-a7a5-7836163a2ac4 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.505146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1075/2156] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:12.506795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.507422 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.507700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.507972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.508662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:30:12.508974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.511112 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.512850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb2df6-44ac-4c9f-9e84-f7dd769699f8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.513329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb2df6-44ac-4c9f-9e84-f7dd769699f8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.513779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb2df6-44ac-4c9f-9e84-f7dd769699f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.524012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:12.528279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1082/2157] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:30:12.535685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.548455 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4aeb2df6-44ac-4c9f-9e84-f7dd769699f8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:12.551966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4aeb2df6-44ac-4c9f-9e84-f7dd769699f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:12.558233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e614893-7946-4c4c-a6d9-90dc8aff4d9c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.565290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1076/2158] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:30:12.569775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.570402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb2df6-44ac-4c9f-9e84-f7dd769699f8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.570772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.571152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.571526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.572309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.581251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.599460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb2df6-44ac-4c9f-9e84-f7dd769699f8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.616368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.617004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.617286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.617634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.618573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:30:12.618877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.623121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.643770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1077/2159] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:30:12.646225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.653052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1083/2160] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:12.658084 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.662920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822a34b-ca3e-434b-a798-df65460d3110 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.793467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5144-416d-48d3-aacb-2a0e24bb6524 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.798954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.799634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.799847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.801175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.801757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.804675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.811672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.812925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.812925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.812925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.813585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.818125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:12.871563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1084/2161] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:12.872605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.880562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1078/2162] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:12.881636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:12.893747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697a38b-b14e-4072-8aa1-b095102fc326 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:12.933195 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:12.986410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:12.987033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:12.987312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:12.987583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:12.988256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:12.989850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.013785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1085/2163] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:13.018019 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:13.042442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f9a7c-ee4d-499f-b5d4-1419962b7e32 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:13.060264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:13.061004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.061288 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.061560 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.062211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:13.064443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.093468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1086/2164] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:13.095437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:13.141691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41356b2c-0870-460d-bfa1-aeee71235486 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:13.150322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26940d0-3c37-424d-ba9d-6046461c6262 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:13.159377 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:13.160015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.160352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.160626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.161723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06531dab-dbde-4298-96d8-a421dcf589c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:13.162495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531dab-dbde-4298-96d8-a421dcf589c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.162872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531dab-dbde-4298-96d8-a421dcf589c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.163268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531dab-dbde-4298-96d8-a421dcf589c0 admin admin] PATH_INFO: `/v3/projects/f291a3a8c0394e928aee758baa470ca7/users/a41fc97ca6ea4a1c8201f068b26e6888/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.163471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:13.170032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.213187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06531dab-dbde-4298-96d8-a421dcf589c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f291a3a8c0394e928aee758baa470ca7, user_id=a41fc97ca6ea4a1c8201f068b26e6888, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:13.219651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06531dab-dbde-4298-96d8-a421dcf589c0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.232286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1079/2165] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:12 2022] POST /identity/v3/users => generated 288 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:13.242315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:13.261661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1087/2166] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:13 2022] PUT /identity/v3/projects/f291a3a8c0394e928aee758baa470ca7/users/a41fc97ca6ea4a1c8201f068b26e6888/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:13.266992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.267271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.267544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.291300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844a270-fabe-44d6-96f1-5fe0fd6c97fc admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:13.324028 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:13.329668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:13.464184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:13.481107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:13.481581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.483776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.484066 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.487107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:13.490480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.496751 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1088/2167] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:30:13.504002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:13.505526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:13.514198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1080/2168] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:30:13.525426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:13.549043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99765345-8bfd-4328-b727-ec0c2a8775d8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:13.603286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:13.604966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.606251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.606774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.609022 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:13.620301 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.654503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1081/2169] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:30:13.667279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:13.678940 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319ddd8-e434-4c98-af6d-a632f8648b78 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:13.688716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cbbe5-cd27-431e-a845-328bcc9f7aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:13.690247 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cbbe5-cd27-431e-a845-328bcc9f7aaa service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.690527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cbbe5-cd27-431e-a845-328bcc9f7aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.690794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cbbe5-cd27-431e-a845-328bcc9f7aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.692591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9347a-ad29-4104-8aec-890adbe16c53 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:13.706938 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:13.708058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.708586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.709130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] PATH_INFO: `/v3/projects/62bfe447f81b4f00869f029157a14d8f/users/b4f3b17582ce4c4bae8a5fcd1288702d/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.713807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cbbe5-cd27-431e-a845-328bcc9f7aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:13.717948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cbbe5-cd27-431e-a845-328bcc9f7aaa service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.757830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1089/2170] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:30:13.761753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62bfe447f81b4f00869f029157a14d8f, user_id=b4f3b17582ce4c4bae8a5fcd1288702d, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:13.768673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:13.793355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1082/2171] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:13 2022] PUT /identity/v3/projects/62bfe447f81b4f00869f029157a14d8f/users/b4f3b17582ce4c4bae8a5fcd1288702d/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:13.813405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c569c75-5425-4204-a391-ab9da984307e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:13.815922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c569c75-5425-4204-a391-ab9da984307e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:13.817431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c569c75-5425-4204-a391-ab9da984307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:13.999009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2c569c75-5425-4204-a391-ab9da984307e None None] MFA Rules not processed for user `b4f3b17582ce4c4bae8a5fcd1288702d`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:14.001785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2c569c75-5425-4204-a391-ab9da984307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:14.090656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:14.196775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1090/2172] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:14.206311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:14.208873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:14.412042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bcade-9d45-41bf-ad38-db671c2663d0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:14.419857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe4c98-9722-4398-9442-a3cfa5f9bb04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:14.420541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe4c98-9722-4398-9442-a3cfa5f9bb04 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:14.420809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe4c98-9722-4398-9442-a3cfa5f9bb04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:14.421076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe4c98-9722-4398-9442-a3cfa5f9bb04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:14.451632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe4c98-9722-4398-9442-a3cfa5f9bb04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:14.457702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe4c98-9722-4398-9442-a3cfa5f9bb04 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:14.489370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1083/2173] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:19.361069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c569c75-5425-4204-a391-ab9da984307e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:19.547060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c569c75-5425-4204-a391-ab9da984307e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:19.560152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:19.560864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:19.561125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:19.561398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] PATH_INFO: `/v3/users/b4f3b17582ce4c4bae8a5fcd1288702d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:19.567565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f3b17582ce4c4bae8a5fcd1288702d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:19.569580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:19.655410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f3b17582ce4c4bae8a5fcd1288702d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:19.673620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f3b17582ce4c4bae8a5fcd1288702d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:19.674514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f3b17582ce4c4bae8a5fcd1288702d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:19.686512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f3b17582ce4c4bae8a5fcd1288702d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:19.711923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f3b17582ce4c4bae8a5fcd1288702d'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:19.727594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1091/2174] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:19 2022] DELETE /identity/v3/users/b4f3b17582ce4c4bae8a5fcd1288702d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:19.732029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:19.732759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:19.732759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:19.767795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:19.771092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:19.887008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1084/2175] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:30:19.889259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:19.890711 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:19.980186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:20.013454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6c33c-37bd-4dab-8bbc-da94e54cee2d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:20.018455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:20.019096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:20.019376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:20.019647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:20.035536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:20.037381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:20.070455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1092/2176] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:20.535695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:20.537906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:20.554081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:20.567065 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11ca43-fa43-48ab-b5b5-c13c343bf24b service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:20.600774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:20.601398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:20.601671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:20.601936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:20.607491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:20.636079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:20.638093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:20.670000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1093/2177] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:30:20.720154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4920b-5440-4954-a575-9fefc8a7e325 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:20.734875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234088f-9d52-40a5-b324-551f807ae547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:20.735515 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234088f-9d52-40a5-b324-551f807ae547 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:20.735902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234088f-9d52-40a5-b324-551f807ae547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:20.736024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234088f-9d52-40a5-b324-551f807ae547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:20.752447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1234088f-9d52-40a5-b324-551f807ae547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:20.795158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:20.902466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234088f-9d52-40a5-b324-551f807ae547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:20.904219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234088f-9d52-40a5-b324-551f807ae547 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:20.934571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1085/2178] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:21.171602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:21.173287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:21.194902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb87b62-4b1e-4cb0-982b-2fa016816704 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:21.209951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:21.210602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:21.210883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:21.211368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] PATH_INFO: `/v3/projects/62bfe447f81b4f00869f029157a14d8f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:21.214495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62bfe447f81b4f00869f029157a14d8f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:21.216578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:21.377401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62bfe447f81b4f00869f029157a14d8f', 'request_id': 'req-45c75b00-b8e0-4e93-875d-0334e672ba17'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:21.397025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62bfe447f81b4f00869f029157a14d8f', 'request_id': 'req-45c75b00-b8e0-4e93-875d-0334e672ba17'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:21.406543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62bfe447f81b4f00869f029157a14d8f', 'request_id': 'req-45c75b00-b8e0-4e93-875d-0334e672ba17'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:21.428536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] The token cache is being invalidate because project 62bfe447f81b4f00869f029157a14d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:21.428916 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:21.429693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-45c75b00-b8e0-4e93-875d-0334e672ba17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:21.440534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1094/2179] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:21 2022] DELETE /identity/v3/projects/62bfe447f81b4f00869f029157a14d8f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:21.445560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d28f5-a645-455b-9995-937ad75944b9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:21.445841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d28f5-a645-455b-9995-937ad75944b9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:21.446113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d28f5-a645-455b-9995-937ad75944b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:21.529912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b73d28f5-a645-455b-9995-937ad75944b9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:21.536632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b73d28f5-a645-455b-9995-937ad75944b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:21.742470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1086/2180] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:21.748212 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f61b-7380-4e2f-bbb2-6dbd2728d1d5 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:21.749395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f61b-7380-4e2f-bbb2-6dbd2728d1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:21.750060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f61b-7380-4e2f-bbb2-6dbd2728d1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:21.817075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-53b5f61b-7380-4e2f-bbb2-6dbd2728d1d5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:21.819375 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-53b5f61b-7380-4e2f-bbb2-6dbd2728d1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:21.855596 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1095/2181] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:21.856390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d28f5-a645-455b-9995-937ad75944b9 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:21.870278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d28f5-a645-455b-9995-937ad75944b9 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:21.883332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:21.884306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:21.887116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:21.887616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:21.888529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:30:21.888997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:21.891025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:21.929528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:21.930167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:21.930863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:21.940167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1087/2182] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:30:21.946382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:21.970723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062105-9d1a-4e75-92b2-20085ef72fd7 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:21.984481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:21.988597 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:21.989723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:21.990220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:21.991705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:21.993932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.027632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] MFA Rules not processed for user `a41fc97ca6ea4a1c8201f068b26e6888`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:22.029199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:22.062572 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1088/2183] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:21 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:22.066079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.205194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee884de-a97d-49d3-a0ec-0899a63062b5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.208062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1096/2184] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:30:22.216125 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.218220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.218425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.218768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.219137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.219786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:22.219914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.222392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.248675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba9a92-7cf9-479f-8151-ed2cea9d10bf None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.255785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.260082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1089/2185] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:22 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:30:22.260620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.260620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.260620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.260620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.261648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.265268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.282126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b1528-0e40-48c1-bcbb-5e250cb8b6c1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.300294 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.301196 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.301543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.301881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.302743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.304620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.312334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1097/2186] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:22 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:22.314716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.317797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:22.327087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1090/2187] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:30:22.332250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.338418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:22.356103 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ad582-6aed-4418-92a1-f95174f99c49 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.440954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d9b84-cf24-48f1-a58b-accb37477220 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.462702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.463789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.464272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.464694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.465494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.468517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.510031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1098/2188] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:22 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:22.513761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.523225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.524144 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.524868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.525341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.526123 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.527748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad02a48-cb37-4d04-abb1-234f417d0c2b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.527997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.545249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1091/2189] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:22.545792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.546498 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.546621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.546899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.547801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.548558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.550250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.572435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83e6f-f8e6-439f-bcda-7996cb843ce0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.577176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1099/2190] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:22.582827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2ab0-9a69-4c4e-8f02-376dfa0e3349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.583652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2ab0-9a69-4c4e-8f02-376dfa0e3349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.584009 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2ab0-9a69-4c4e-8f02-376dfa0e3349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.584449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2ab0-9a69-4c4e-8f02-376dfa0e3349 admin admin] PATH_INFO: `/v3/projects/9f309e5e9bf84fff939c088e7c256bfb/users/261dfac380ce4eb28eb197cd512b6edc/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.584921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.618045 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2ab0-9a69-4c4e-8f02-376dfa0e3349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f309e5e9bf84fff939c088e7c256bfb, user_id=261dfac380ce4eb28eb197cd512b6edc, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.623245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79573c93-3fbb-463b-b33a-12d3dcfe6493 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.624874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2ab0-9a69-4c4e-8f02-376dfa0e3349 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.637594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.638494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.640847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.641513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] PATH_INFO: `/v3/projects/5c38edf003e74119a853a203be7a2fd8/users/bb199ca4c05e4c7fbf759abf4d23050c/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.666091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1092/2191] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:22 2022] PUT /identity/v3/projects/9f309e5e9bf84fff939c088e7c256bfb/users/261dfac380ce4eb28eb197cd512b6edc/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:22.668867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.670682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.671154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.682585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c38edf003e74119a853a203be7a2fd8, user_id=bb199ca4c05e4c7fbf759abf4d23050c, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.685676 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.715569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:22.723452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:22.736340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1100/2192] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:22 2022] PUT /identity/v3/projects/5c38edf003e74119a853a203be7a2fd8/users/bb199ca4c05e4c7fbf759abf4d23050c/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:22.741016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.874614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccfdb6-a810-4d72-8ac7-1a90c0c4202c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.886295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.886919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.887192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.887467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.888342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.890516 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.916040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1101/2193] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:22.917178 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.918777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1093/2194] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:22.930219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:22.931904 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:22.940909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734793e8-b63a-434b-be4f-5b60358e644b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:22.954495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b72478-cd44-4cbb-8185-a41d94743d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:22.955347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72478-cd44-4cbb-8185-a41d94743d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:22.955521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72478-cd44-4cbb-8185-a41d94743d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:22.955911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72478-cd44-4cbb-8185-a41d94743d28 admin admin] PATH_INFO: `/v3/projects/5c38edf003e74119a853a203be7a2fd8/users/bb199ca4c05e4c7fbf759abf4d23050c/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:22.959193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b72478-cd44-4cbb-8185-a41d94743d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c38edf003e74119a853a203be7a2fd8, user_id=bb199ca4c05e4c7fbf759abf4d23050c, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:22.963535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b72478-cd44-4cbb-8185-a41d94743d28 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:22.992162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1102/2195] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:22 2022] PUT /identity/v3/projects/5c38edf003e74119a853a203be7a2fd8/users/bb199ca4c05e4c7fbf759abf4d23050c/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:23.127065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c8d90-6b6d-424b-bb87-de72da0d3bfb None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:23.207821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:23.291094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:23.291730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:23.292011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:23.292310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:23.503430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:23.504927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:23.533546 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1094/2196] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:27.222646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:27.223433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:27.223433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:27.300431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] MFA Rules not processed for user `261dfac380ce4eb28eb197cd512b6edc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:27.301883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:27.456308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:27.518022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1103/2197] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:27.530911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:27.533557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:27.725436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8d4cd-6bbd-4d96-9d8e-f0881945a965 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:27.744526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:27.746267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:27.746556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:27.746815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:27.762074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:27.763730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:27.780841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1095/2198] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:28.692672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:28.694730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:28.760515 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eed12-1b87-41b4-9afd-b518e62fb386 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:28.797399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:28.798415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:28.799092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:28.799614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] PATH_INFO: `/v3/users/8f0bb483c548429295c02558520c6898` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:28.801222 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f0bb483c548429295c02558520c6898)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:28.803494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:28.908099 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f0bb483c548429295c02558520c6898'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:28.930363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f0bb483c548429295c02558520c6898'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:28.931126 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f0bb483c548429295c02558520c6898'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:28.941096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f0bb483c548429295c02558520c6898'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:28.956686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-cc91719c-9df4-401c-b4ef-adaa49deeb9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f0bb483c548429295c02558520c6898'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:28.992894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1104/2199] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:28 2022] DELETE /identity/v3/users/8f0bb483c548429295c02558520c6898 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:29.396529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:29.416080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97615e-e836-4d30-a166-f9535a5dae12 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:29.501079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:29.501632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:29.501983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:29.502277 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] PATH_INFO: `/v3/projects/5d1a967f9b62401ba95d65a2747e3ec0` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:29.504423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1a967f9b62401ba95d65a2747e3ec0)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:29.506490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:29.586694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:29.632520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1a967f9b62401ba95d65a2747e3ec0', 'request_id': 'req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:29.646470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1a967f9b62401ba95d65a2747e3ec0', 'request_id': 'req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:29.655168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ed77-da9d-4795-832e-0ca1dc776260 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:29.658780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ed77-da9d-4795-832e-0ca1dc776260 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:29.660165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ed77-da9d-4795-832e-0ca1dc776260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:29.697192 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1a967f9b62401ba95d65a2747e3ec0', 'request_id': 'req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:29.727772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2177ed77-da9d-4795-832e-0ca1dc776260 None None] MFA Rules not processed for user `261dfac380ce4eb28eb197cd512b6edc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:29.730108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2177ed77-da9d-4795-832e-0ca1dc776260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:29.745978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] The token cache is being invalidate because project 5d1a967f9b62401ba95d65a2747e3ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:29.748972 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:29.757842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-369ef838-c081-4420-ae7b-e6e4fd0ab4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:29.768359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1096/2200] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:29 2022] DELETE /identity/v3/projects/5d1a967f9b62401ba95d65a2747e3ec0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:29.773167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d065-8e23-44e7-9aad-b0ba863e5e7c None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:29.774788 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d065-8e23-44e7-9aad-b0ba863e5e7c None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:29.775293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d065-8e23-44e7-9aad-b0ba863e5e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:29.846015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5719d065-8e23-44e7-9aad-b0ba863e5e7c None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:29.850802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5719d065-8e23-44e7-9aad-b0ba863e5e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:29.883229 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1105/2201] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:30:30.147387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1097/2202] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:30.149367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44684b-3ecf-40d9-8259-59a07d447419 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:30.149776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44684b-3ecf-40d9-8259-59a07d447419 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:30.150175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44684b-3ecf-40d9-8259-59a07d447419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:30.242570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-da44684b-3ecf-40d9-8259-59a07d447419 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:30.244858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-da44684b-3ecf-40d9-8259-59a07d447419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:30.264156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1106/2203] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:30.270111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719d065-8e23-44e7-9aad-b0ba863e5e7c None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:30.289825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719d065-8e23-44e7-9aad-b0ba863e5e7c None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:30.299495 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:30.300203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:30.302368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:30.302368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:30.304137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:30:30.304471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:30.311245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:30.349700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44684b-3ecf-40d9-8259-59a07d447419 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:30.366850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1098/2204] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:30:30.369655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44684b-3ecf-40d9-8259-59a07d447419 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:30.383739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:30.386599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:30.388368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:30.388669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:30.391181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:30.397580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:30.482890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:30.494071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1107/2205] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:30 2022] POST /identity/v3/projects => generated 403 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:30.501003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6811fa-77db-4f1f-b7dc-59a3b44ad4d3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:30.722586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:30.723212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:30.723504 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:30.723832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:30.724850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:30.735995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:30.799243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1099/2206] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:30 2022] POST /identity/v3/users => generated 302 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:30.800623 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:30.820639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e74d2-d734-41e1-9dd8-0fc60f238ac8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:30.845647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:30.848357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:30.848663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:30.848960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:30.849774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:30.851647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:30.885234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1108/2207] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:30:30.886768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:30.908577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6323f3-fbe9-40b0-a336-55f75f87115b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:30.943700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:30.944875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:30.945269 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:30.945657 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:30.946521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:30.951602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:31.015730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1100/2208] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:30:31.018190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:31.030749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac382ff-54fa-4d84-90ef-0c7467799730 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:31.052631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5f5b4-52c7-48fa-bd2b-876c7346e82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:31.055470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5f5b4-52c7-48fa-bd2b-876c7346e82d admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:31.055752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5f5b4-52c7-48fa-bd2b-876c7346e82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:31.056031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5f5b4-52c7-48fa-bd2b-876c7346e82d admin admin] PATH_INFO: `/v3/projects/8d8479eb60b545af8bd2f294bcb054ce/users/2d4fc74df56b4f73a3eb12696fa51786/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:31.138617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:31.147582 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5f5b4-52c7-48fa-bd2b-876c7346e82d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d8479eb60b545af8bd2f294bcb054ce, user_id=2d4fc74df56b4f73a3eb12696fa51786, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:31.157933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:31.160458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:31.172719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5f5b4-52c7-48fa-bd2b-876c7346e82d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:31.283682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1109/2209] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:31 2022] PUT /identity/v3/projects/8d8479eb60b545af8bd2f294bcb054ce/users/2d4fc74df56b4f73a3eb12696fa51786/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:31.291918 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:31.295018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00ff88-a755-447e-bbf1-0faf2a788619 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:31.295871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00ff88-a755-447e-bbf1-0faf2a788619 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:31.295871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00ff88-a755-447e-bbf1-0faf2a788619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:31.473377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4a00ff88-a755-447e-bbf1-0faf2a788619 None None] MFA Rules not processed for user `2d4fc74df56b4f73a3eb12696fa51786`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:31.474658 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4a00ff88-a755-447e-bbf1-0faf2a788619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:31.540321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31bf06-23fb-4937-867c-65097f98ad78 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:31.805376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:31.808150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:31.808633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:31.809881 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:31.817162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:31.824245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1110/2210] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:31.826531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00ff88-a755-447e-bbf1-0faf2a788619 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:31.875548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:31.886478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:31.922770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1101/2211] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:31.985454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00ff88-a755-447e-bbf1-0faf2a788619 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:32.010640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:32.012324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:32.012614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:32.012873 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:32.013523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:32.016060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:32.123254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1111/2212] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:31 2022] POST /identity/v3/projects => generated 405 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:32.129805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:32.149773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d07f96-9bda-4e7e-8a4e-6d51f4faef46 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:32.322232 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:32.323060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:32.323327 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:32.323601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:32.324273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:32.326460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:32.390100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1102/2213] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:32 2022] POST /identity/v3/users => generated 309 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:32.391021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:32.422681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6102a-ace8-473d-abf8-4e2b529560f2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:32.445472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:32.445697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:32.445964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:32.448077 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:32.456948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:32.458292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:32.492830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1112/2214] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:30:32.501950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:32.519166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9877ec-387c-4eda-9b55-931b26769ec1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:32.540629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:32.543227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:32.543511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:32.550945 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] PATH_INFO: `/v3/projects/40622c67476c4e7c97fa1dd80f561c5e/users/98dfcbbb82ce47b996a7d96d41256439/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:32.648040 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40622c67476c4e7c97fa1dd80f561c5e, user_id=98dfcbbb82ce47b996a7d96d41256439, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:32.662362 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:32.767887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:32.770218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1103/2215] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:32 2022] PUT /identity/v3/projects/40622c67476c4e7c97fa1dd80f561c5e/users/98dfcbbb82ce47b996a7d96d41256439/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:32.810951 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f841ab7-cb63-44d7-b4f7-c6fbf1d31204 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:33.145289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:33.168504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aa111-c196-4585-9ace-9aacb42262c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:33.169297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa111-c196-4585-9ace-9aacb42262c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:33.169577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa111-c196-4585-9ace-9aacb42262c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:33.173967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa111-c196-4585-9ace-9aacb42262c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:33.178948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aa111-c196-4585-9ace-9aacb42262c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:33.186175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aa111-c196-4585-9ace-9aacb42262c6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:33.282347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1113/2216] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:33.284403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:33.310915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77f47-7378-44cd-bc1c-5f9c80901b64 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:33.343416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:33.344465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:33.344969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:33.346283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] PATH_INFO: `/v3/projects/40622c67476c4e7c97fa1dd80f561c5e/users/98dfcbbb82ce47b996a7d96d41256439/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:33.353651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] RBAC: Authorizing `identity:create_grant(project_id=40622c67476c4e7c97fa1dd80f561c5e, user_id=98dfcbbb82ce47b996a7d96d41256439, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:33.363756 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:33.415368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe272-4f24-4e9b-9fa3-e5a6091a5d64 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:33.415776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe272-4f24-4e9b-9fa3-e5a6091a5d64 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:33.416170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe272-4f24-4e9b-9fa3-e5a6091a5d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:33.421137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1104/2217] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:33 2022] PUT /identity/v3/projects/40622c67476c4e7c97fa1dd80f561c5e/users/98dfcbbb82ce47b996a7d96d41256439/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:33.506803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ce3fe272-4f24-4e9b-9fa3-e5a6091a5d64 None None] MFA Rules not processed for user `98dfcbbb82ce47b996a7d96d41256439`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:33.507570 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ce3fe272-4f24-4e9b-9fa3-e5a6091a5d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:33.916111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:33.917927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:33.931845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1114/2218] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:34.085129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:34.178089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860ae6e-c6b6-4e13-84eb-92801530112f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:34.213085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b008-cae7-4f3e-9301-220b419d39cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:34.214203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b008-cae7-4f3e-9301-220b419d39cd service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:34.214537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b008-cae7-4f3e-9301-220b419d39cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:34.214741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b008-cae7-4f3e-9301-220b419d39cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:34.285140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b008-cae7-4f3e-9301-220b419d39cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:34.288825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b008-cae7-4f3e-9301-220b419d39cd service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:34.362517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1105/2219] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:33 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:34.666374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe272-4f24-4e9b-9fa3-e5a6091a5d64 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:35.037738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe272-4f24-4e9b-9fa3-e5a6091a5d64 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:35.049864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:35.050905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:35.054463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:35.054463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] PATH_INFO: `/v3/users/2d4fc74df56b4f73a3eb12696fa51786` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:35.056344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d4fc74df56b4f73a3eb12696fa51786)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:35.076395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:35.245874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d4fc74df56b4f73a3eb12696fa51786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:35.304451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d4fc74df56b4f73a3eb12696fa51786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:35.310565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d4fc74df56b4f73a3eb12696fa51786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:35.326268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d4fc74df56b4f73a3eb12696fa51786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:35.364973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:35.393539 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d4fc74df56b4f73a3eb12696fa51786'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:35.418350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1115/2220] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:34 2022] DELETE /identity/v3/users/2d4fc74df56b4f73a3eb12696fa51786 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:35.420156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:35.420730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:35.420730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:35.457277 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:35.461666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:35.710289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1106/2221] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:35.723708 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:35.725268 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:35.839950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3e562-b4a0-4b49-a9ef-2cbcce0262ae admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:35.846666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:35.849145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:35.849756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:35.850305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:35.873588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:35.877319 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:35.930831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1116/2222] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:37.694036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:37.697065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:37.715864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdc171-61f0-483a-ba0c-7521483b4bab None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:37.738250 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:37.739966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:37.740576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:37.744365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] PATH_INFO: `/v3/projects/8d8479eb60b545af8bd2f294bcb054ce` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:37.749127 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d8479eb60b545af8bd2f294bcb054ce)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:37.757654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:37.908857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d8479eb60b545af8bd2f294bcb054ce', 'request_id': 'req-7d68e59f-6bb9-4958-b91c-f71015a80c68'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:37.932150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d8479eb60b545af8bd2f294bcb054ce', 'request_id': 'req-7d68e59f-6bb9-4958-b91c-f71015a80c68'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:37.943700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d8479eb60b545af8bd2f294bcb054ce', 'request_id': 'req-7d68e59f-6bb9-4958-b91c-f71015a80c68'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:37.972073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] The token cache is being invalidate because project 8d8479eb60b545af8bd2f294bcb054ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:37.973525 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:37.992883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7d68e59f-6bb9-4958-b91c-f71015a80c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:37.995705 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1107/2223] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:37 2022] DELETE /identity/v3/projects/8d8479eb60b545af8bd2f294bcb054ce => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:38.058078 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:38.085639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:38.085934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:38.086967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:38.094136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828dda-4cf5-4184-873f-16ab5bd28de3 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:38.290333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] MFA Rules not processed for user `bb199ca4c05e4c7fbf759abf4d23050c`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:38.291827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:38.390725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:38.392072 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:38.392451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:38.392979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] PATH_INFO: `/v3/users/98dfcbbb82ce47b996a7d96d41256439` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:38.394361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98dfcbbb82ce47b996a7d96d41256439)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:38.403550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:38.494619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:38.560217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98dfcbbb82ce47b996a7d96d41256439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:38.585324 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98dfcbbb82ce47b996a7d96d41256439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:38.588061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98dfcbbb82ce47b996a7d96d41256439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:38.604367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1108/2224] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:38.630400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98dfcbbb82ce47b996a7d96d41256439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:38.664131 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98dfcbbb82ce47b996a7d96d41256439'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:38.694088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1117/2225] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:38 2022] DELETE /identity/v3/users/98dfcbbb82ce47b996a7d96d41256439 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:38.755848 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:38.757440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:39.007353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd524b-db3d-4719-949e-634d2ffd1090 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:39.154289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed4e32-976a-4c49-824a-b7d9b8e410bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:39.156338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4e32-976a-4c49-824a-b7d9b8e410bf service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:39.156625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4e32-976a-4c49-824a-b7d9b8e410bf service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:39.157932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4e32-976a-4c49-824a-b7d9b8e410bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:39.160658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-68ed4e32-976a-4c49-824a-b7d9b8e410bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:39.289355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed4e32-976a-4c49-824a-b7d9b8e410bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:39.291859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed4e32-976a-4c49-824a-b7d9b8e410bf service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:39.314163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1109/2226] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:40.029591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:40.031633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:40.118736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:40.304724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d687ac-842a-4a27-a54b-774ae16c2fb2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:40.309681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:40.310336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:40.311620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:40.312440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] PATH_INFO: `/v3/projects/40622c67476c4e7c97fa1dd80f561c5e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:40.313762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40622c67476c4e7c97fa1dd80f561c5e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:40.316214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:40.409619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40622c67476c4e7c97fa1dd80f561c5e', 'request_id': 'req-12285376-4fb6-4185-b0d3-4b7786f324f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:40.429364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40622c67476c4e7c97fa1dd80f561c5e', 'request_id': 'req-12285376-4fb6-4185-b0d3-4b7786f324f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:40.442118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40622c67476c4e7c97fa1dd80f561c5e', 'request_id': 'req-12285376-4fb6-4185-b0d3-4b7786f324f5'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:40.475202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] The token cache is being invalidate because project 40622c67476c4e7c97fa1dd80f561c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:40.475769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:40.476632 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-12285376-4fb6-4185-b0d3-4b7786f324f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:40.482548 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1118/2227] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:40 2022] DELETE /identity/v3/projects/40622c67476c4e7c97fa1dd80f561c5e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:40.484120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f4d7-5653-4829-a252-4a2fcf4f5062 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:40.484630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f4d7-5653-4829-a252-4a2fcf4f5062 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:40.485193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f4d7-5653-4829-a252-4a2fcf4f5062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:40.531159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1040f4d7-5653-4829-a252-4a2fcf4f5062 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:40.537461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1040f4d7-5653-4829-a252-4a2fcf4f5062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:40.588017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:40.727957 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1110/2228] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:40.730179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d648c0-d38a-4b36-80d9-3699b74c1b7e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:40.733260 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d648c0-d38a-4b36-80d9-3699b74c1b7e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:40.736096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d648c0-d38a-4b36-80d9-3699b74c1b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:40.771890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a2d648c0-d38a-4b36-80d9-3699b74c1b7e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:40.773219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a2d648c0-d38a-4b36-80d9-3699b74c1b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:40.787773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1119/2229] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:40.789046 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f4d7-5653-4829-a252-4a2fcf4f5062 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:40.803852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f4d7-5653-4829-a252-4a2fcf4f5062 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:40.810383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:40.811470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:40.812047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:40.812680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:40.813685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:30:40.814151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:40.817388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:40.839943 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1111/2230] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:30:40.844466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d648c0-d38a-4b36-80d9-3699b74c1b7e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:40.855542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d648c0-d38a-4b36-80d9-3699b74c1b7e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:40.864312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:40.865022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:40.865393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:40.866157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:40.866777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:40.869940 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:40.915958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1120/2231] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:40 2022] POST /identity/v3/projects => generated 387 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:40.917080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:40.932063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d8790-0b70-47f8-906c-be861a2f4289 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:41.072628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:41.073468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:41.073648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:41.074582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:41.074714 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:41.081536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:41.126713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1112/2232] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:40 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:41.130512 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:41.159908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5048b-a926-4cc9-af3f-31a7c0627651 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:41.188822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:41.189419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:41.189769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:41.190122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:41.190831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:41.199748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:41.221358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1121/2233] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:41.222785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:41.239516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ca44e-ed87-4007-befa-670f285e01b3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:41.253645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8583d2-0f68-4d4c-9f97-50bd780febe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:41.254439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8583d2-0f68-4d4c-9f97-50bd780febe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:41.254930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8583d2-0f68-4d4c-9f97-50bd780febe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:41.255115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8583d2-0f68-4d4c-9f97-50bd780febe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:41.255856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8583d2-0f68-4d4c-9f97-50bd780febe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:41.257901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8583d2-0f68-4d4c-9f97-50bd780febe8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:41.280793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1113/2234] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:41.284584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:41.317220 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16d22-5934-497b-949c-ff89c98c8496 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:41.322171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:41.322870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:41.323145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:41.323417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] PATH_INFO: `/v3/projects/55cfc4ab7e004e1e81d635a4797acc1f/users/7ae5992c601846e9bb4244f74b227e3a/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:41.356856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cfc4ab7e004e1e81d635a4797acc1f, user_id=7ae5992c601846e9bb4244f74b227e3a, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:41.361135 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:41.385917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1122/2235] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:41 2022] PUT /identity/v3/projects/55cfc4ab7e004e1e81d635a4797acc1f/users/7ae5992c601846e9bb4244f74b227e3a/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:41.398467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511451e-b2d2-4f11-9e25-20fe0182fa75 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:41.398906 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511451e-b2d2-4f11-9e25-20fe0182fa75 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:41.401358 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511451e-b2d2-4f11-9e25-20fe0182fa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:41.491741 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2511451e-b2d2-4f11-9e25-20fe0182fa75 None None] MFA Rules not processed for user `7ae5992c601846e9bb4244f74b227e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:41.494755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2511451e-b2d2-4f11-9e25-20fe0182fa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:41.693262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1114/2236] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:41.694012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:41.824052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:41.863390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8bff9-4c3f-439d-8c0e-629455f0b3d6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:41.878491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:41.879155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:41.879405 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:41.879709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:41.880448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:41.882355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:41.948275 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1123/2237] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:41 2022] POST /identity/v3/projects => generated 387 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:30:41.952548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511451e-b2d2-4f11-9e25-20fe0182fa75 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:41.975447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511451e-b2d2-4f11-9e25-20fe0182fa75 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:42.003922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:42.135169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:42.136076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:42.136723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:42.137152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:42.138152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:42.148289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:42.203054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1115/2238] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:41 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:42.207254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:42.224259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369cd3-7989-4903-8b63-3444c4e27d63 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:42.244497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:42.245393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:42.246874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:42.247321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:42.250925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:42.253087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:42.269583 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1124/2239] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:42.272766 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:42.288995 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a166a-818a-40c1-b519-a6f75888af5d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:42.310032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:42.314138 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:42.317832 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:42.319004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] PATH_INFO: `/v3/projects/3ba94560f64c46a3833496f4b0798c68/users/1b24509b0f7f41beac7b6243e49aeedd/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:42.399820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba94560f64c46a3833496f4b0798c68, user_id=1b24509b0f7f41beac7b6243e49aeedd, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:42.409129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:42.467195 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:42.488594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1116/2240] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:42 2022] PUT /identity/v3/projects/3ba94560f64c46a3833496f4b0798c68/users/1b24509b0f7f41beac7b6243e49aeedd/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:30:42.502741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32bc0-08e4-4e1e-a666-587593267610 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:42.809083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc54fda-af9d-4921-a25a-f579ab2b1625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:42.809718 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc54fda-af9d-4921-a25a-f579ab2b1625 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:42.810001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc54fda-af9d-4921-a25a-f579ab2b1625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:42.810262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc54fda-af9d-4921-a25a-f579ab2b1625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:42.813527 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc54fda-af9d-4921-a25a-f579ab2b1625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:42.818177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc54fda-af9d-4921-a25a-f579ab2b1625 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:42.850332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1125/2241] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:42.857470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:42.886531 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ba01-6b96-4f88-b646-e3c8546ebf08 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:42.915595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:42.917037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:42.917390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:42.917672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] PATH_INFO: `/v3/projects/3ba94560f64c46a3833496f4b0798c68/users/1b24509b0f7f41beac7b6243e49aeedd/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:42.924573 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba94560f64c46a3833496f4b0798c68, user_id=1b24509b0f7f41beac7b6243e49aeedd, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:42.927407 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:42.953734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1117/2242] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:42 2022] PUT /identity/v3/projects/3ba94560f64c46a3833496f4b0798c68/users/1b24509b0f7f41beac7b6243e49aeedd/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:42.964321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e56bf-11da-49d7-b91b-b910618b6660 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:42.964668 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e56bf-11da-49d7-b91b-b910618b6660 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:42.964887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e56bf-11da-49d7-b91b-b910618b6660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:43.037489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-4b0e56bf-11da-49d7-b91b-b910618b6660 None None] MFA Rules not processed for user `1b24509b0f7f41beac7b6243e49aeedd`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:43.038794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4b0e56bf-11da-49d7-b91b-b910618b6660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:43.292882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1126/2243] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:43.320721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:43.326701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:43.597478 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621bdf2-2a1d-450c-bb00-4dbe75be9717 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:43.625418 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a92d9-2a02-4fc8-9eff-f82a2f1029c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:43.626083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a92d9-2a02-4fc8-9eff-f82a2f1029c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:43.626370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a92d9-2a02-4fc8-9eff-f82a2f1029c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:43.626640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a92d9-2a02-4fc8-9eff-f82a2f1029c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:43.652942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a92d9-2a02-4fc8-9eff-f82a2f1029c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:43.655976 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a92d9-2a02-4fc8-9eff-f82a2f1029c0 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:43.724116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1118/2244] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:43.779419 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e56bf-11da-49d7-b91b-b910618b6660 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:43.842053 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:43.870961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e56bf-11da-49d7-b91b-b910618b6660 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:43.878691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:43.880080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:43.880446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:43.880825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] PATH_INFO: `/v3/users/7ae5992c601846e9bb4244f74b227e3a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:43.882503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae5992c601846e9bb4244f74b227e3a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:43.885199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:43.941095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:43.941975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:43.942547 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:43.963278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] MFA Rules not processed for user `a385ba4ea6ee4a2fa76f0f12719427ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:43.964562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:43.964709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae5992c601846e9bb4244f74b227e3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:43.985599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae5992c601846e9bb4244f74b227e3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:43.986746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae5992c601846e9bb4244f74b227e3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:44.000154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae5992c601846e9bb4244f74b227e3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:44.015103 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-c9bdbd40-1854-45c4-9c25-a4c1f78931ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae5992c601846e9bb4244f74b227e3a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:44.023095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1127/2245] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:43 2022] DELETE /identity/v3/users/7ae5992c601846e9bb4244f74b227e3a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:44.028062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765692e-b44c-415f-acda-94246736f8ce None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:44.028638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765692e-b44c-415f-acda-94246736f8ce None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:44.029017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765692e-b44c-415f-acda-94246736f8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:44.053745 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-1765692e-b44c-415f-acda-94246736f8ce None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:44.055050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1765692e-b44c-415f-acda-94246736f8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:44.056304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1119/2246] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:30:44.141357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1128/2247] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:30:44.151892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:44.153627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:44.177721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:44.229821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af326bf-4332-46b7-a19d-d20fa3123700 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:44.296116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:44.296875 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:44.297154 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:44.297521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:44.308910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:44.310567 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:44.324392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1120/2248] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:45.191667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765692e-b44c-415f-acda-94246736f8ce None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:45.194866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-1765692e-b44c-415f-acda-94246736f8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:45.250922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765692e-b44c-415f-acda-94246736f8ce None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:45.289647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:45.290300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:45.290569 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:45.290834 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] PATH_INFO: `/v3/projects/55cfc4ab7e004e1e81d635a4797acc1f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:45.292079 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cfc4ab7e004e1e81d635a4797acc1f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:45.294093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:45.408923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cfc4ab7e004e1e81d635a4797acc1f', 'request_id': 'req-ccffc1dc-2335-42ae-85f2-1a4d740f9045'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.414427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:45.426523 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cfc4ab7e004e1e81d635a4797acc1f', 'request_id': 'req-ccffc1dc-2335-42ae-85f2-1a4d740f9045'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.437403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cfc4ab7e004e1e81d635a4797acc1f', 'request_id': 'req-ccffc1dc-2335-42ae-85f2-1a4d740f9045'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.455339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] The token cache is being invalidate because project 55cfc4ab7e004e1e81d635a4797acc1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:45.455732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.464458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.464661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:45.473627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1129/2249] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:45 2022] DELETE /identity/v3/projects/55cfc4ab7e004e1e81d635a4797acc1f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:45.478281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003b94-ce44-4a41-84d0-a0ceaeac82c1 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:45.550821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:45.552782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:45.553316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:45.553910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] PATH_INFO: `/v3/users/1b24509b0f7f41beac7b6243e49aeedd` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:45.556341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b24509b0f7f41beac7b6243e49aeedd)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:45.558917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:45.663835 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b24509b0f7f41beac7b6243e49aeedd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.683629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b24509b0f7f41beac7b6243e49aeedd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.684460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b24509b0f7f41beac7b6243e49aeedd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.706140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:45.711032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b24509b0f7f41beac7b6243e49aeedd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.728499 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-4846d2d6-30b3-4ce0-b0eb-75c3dd01440e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b24509b0f7f41beac7b6243e49aeedd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:45.740927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1121/2250] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:30:45 2022] DELETE /identity/v3/users/1b24509b0f7f41beac7b6243e49aeedd => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:46.194542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:46.196563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:46.226372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffc1dc-2335-42ae-85f2-1a4d740f9045 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:46.369451 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:46.370383 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:46.371186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:46.371323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] PATH_INFO: `/v3/projects/3ba94560f64c46a3833496f4b0798c68` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:46.374279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ba94560f64c46a3833496f4b0798c68)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:46.379236 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:46.459103 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba94560f64c46a3833496f4b0798c68', 'request_id': 'req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:46.478468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba94560f64c46a3833496f4b0798c68', 'request_id': 'req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:46.486417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba94560f64c46a3833496f4b0798c68', 'request_id': 'req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:46.504888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] The token cache is being invalidate because project 3ba94560f64c46a3833496f4b0798c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:30:46.505334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:46.508050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b0aaa4e4-d464-4b22-9a88-6ee502ad87bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:30:46.519728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fc699-a806-4ba7-a788-5eda70acdefc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:46.520662 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fc699-a806-4ba7-a788-5eda70acdefc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:46.521202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fc699-a806-4ba7-a788-5eda70acdefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:46.527210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1130/2251] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:30:46 2022] DELETE /identity/v3/projects/3ba94560f64c46a3833496f4b0798c68 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:30:46.600779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0f5fc699-a806-4ba7-a788-5eda70acdefc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:46.602043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0f5fc699-a806-4ba7-a788-5eda70acdefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:46.740140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1122/2252] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:46.749216 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81955-4447-41e3-b089-7ee56936bd9b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:46.750052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81955-4447-41e3-b089-7ee56936bd9b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:46.750528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81955-4447-41e3-b089-7ee56936bd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:46.773688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-79f81955-4447-41e3-b089-7ee56936bd9b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:46.775035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-79f81955-4447-41e3-b089-7ee56936bd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:46.785393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1131/2253] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:46.791137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fc699-a806-4ba7-a788-5eda70acdefc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:46.804624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fc699-a806-4ba7-a788-5eda70acdefc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:46.810690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:46.811902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:46.812742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:46.813021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:46.813820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:30:46.814065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:46.816303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:46.841978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1123/2254] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:30:46.846896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f81955-4447-41e3-b089-7ee56936bd9b None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:46.860665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f81955-4447-41e3-b089-7ee56936bd9b None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:46.867332 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:46.867988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:46.868322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:46.868601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:46.869389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:46.871520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:46.910148 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1132/2255] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:30:46 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:30:46.911289 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:46.924502 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfa254-95ca-403a-b3e9-1cd8f24e760e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:47.044317 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:47.045570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:47.046621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:47.046621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:47.047479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:47.051201 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:47.088182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1124/2256] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:30:46 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:30:47.097224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:47.113060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae08f79-c061-4db5-b0ae-0711adef6c81 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:47.127826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:47.128755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:47.129041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:47.129434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:47.130287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:47.132824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:47.152147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1133/2257] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:47.153454 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:47.177973 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af706df3-e9ac-444d-af86-a3827f699a47 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:47.185661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47097fb-9da9-4ad6-a98f-ba217398775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:47.186731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47097fb-9da9-4ad6-a98f-ba217398775a admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:47.187011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47097fb-9da9-4ad6-a98f-ba217398775a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:47.187290 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47097fb-9da9-4ad6-a98f-ba217398775a admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:47.187974 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47097fb-9da9-4ad6-a98f-ba217398775a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:47.189814 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47097fb-9da9-4ad6-a98f-ba217398775a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:47.209309 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1125/2258] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:30:47.210921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:47.224682 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871dfde-b758-4247-bf49-a170b26dc186 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:47.230931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:47.231715 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:47.231992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:47.232291 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] PATH_INFO: `/v3/projects/1883384b97484ce1b0b1d9dc58df9930/users/8847c39f8cc34bf1845615feba3dc5cf/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:47.260251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1883384b97484ce1b0b1d9dc58df9930, user_id=8847c39f8cc34bf1845615feba3dc5cf, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:47.267549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:47.288825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1134/2259] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:30:47 2022] PUT /identity/v3/projects/1883384b97484ce1b0b1d9dc58df9930/users/8847c39f8cc34bf1845615feba3dc5cf/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:30:47.299350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f6eb9-9241-4505-ad77-51dc09343999 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:47.299630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f6eb9-9241-4505-ad77-51dc09343999 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:47.299910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f6eb9-9241-4505-ad77-51dc09343999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:47.345243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-5d7f6eb9-9241-4505-ad77-51dc09343999 None None] MFA Rules not processed for user `8847c39f8cc34bf1845615feba3dc5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:47.347205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-5d7f6eb9-9241-4505-ad77-51dc09343999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:47.416343 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:47.481268 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1126/2260] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:30:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:47.502549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:47.504410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:47.522839 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:47.621894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3935c-5790-45fe-a216-466a505de6c8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:47.636468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:47.637159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:47.637382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:47.637650 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:47.650333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:47.652001 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:47.662838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1135/2261] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:30:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:30:49.658663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad034e88-4a3d-46c3-8eb5-c56039da324b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:49.659208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad034e88-4a3d-46c3-8eb5-c56039da324b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:49.659208 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad034e88-4a3d-46c3-8eb5-c56039da324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:49.685065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ad034e88-4a3d-46c3-8eb5-c56039da324b None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:30:49.686316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ad034e88-4a3d-46c3-8eb5-c56039da324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:49.782582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1127/2262] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:30:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:30:49.799755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:30:49.803028 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:30:49.905237 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7675-64e1-4990-a3ae-788c35408925 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:30:49.910305 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:30:49.910912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:30:49.911189 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:30:49.911458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:30:49.920310 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:30:49.921879 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:30:49.930559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1136/2263] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:30:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:30:59.775124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=32 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:30:59.926954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=54 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:04.831145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274079e-efd2-4978-923b-a1ced40e605e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:04.831722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274079e-efd2-4978-923b-a1ced40e605e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:04.831722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274079e-efd2-4978-923b-a1ced40e605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:04.851634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d274079e-efd2-4978-923b-a1ced40e605e None None] MFA Rules not processed for user `a41fc97ca6ea4a1c8201f068b26e6888`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:04.853194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d274079e-efd2-4978-923b-a1ced40e605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:04.909726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1128/2264] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:09.647384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:09.728914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edf349-154a-4c86-aee4-fadef2aa0965 service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:09.735241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:09.736133 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:09.736619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:09.737060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] PATH_INFO: `/v3/users/8847c39f8cc34bf1845615feba3dc5cf` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:09.738390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8847c39f8cc34bf1845615feba3dc5cf)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:09.740544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:09.802784 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8847c39f8cc34bf1845615feba3dc5cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:09.818391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8847c39f8cc34bf1845615feba3dc5cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:09.819385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8847c39f8cc34bf1845615feba3dc5cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:09.829412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8847c39f8cc34bf1845615feba3dc5cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:09.843999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8847c39f8cc34bf1845615feba3dc5cf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:09.852033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1137/2265] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:31:09 2022] DELETE /identity/v3/users/8847c39f8cc34bf1845615feba3dc5cf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:31:09.857095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:09.857953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:09.858387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:09.879511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:09.881098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:09.977447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1129/2266] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:09.985360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:09.987781 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:10.139185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd715a7-ca78-4057-bec1-096587fe026e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:10.151021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29054af2-ce98-4121-b1de-d1de1ee6bdf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:10.151739 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29054af2-ce98-4121-b1de-d1de1ee6bdf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:10.152096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29054af2-ce98-4121-b1de-d1de1ee6bdf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:10.152459 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29054af2-ce98-4121-b1de-d1de1ee6bdf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:10.176460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29054af2-ce98-4121-b1de-d1de1ee6bdf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:10.180410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29054af2-ce98-4121-b1de-d1de1ee6bdf2 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:10.202503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1138/2267] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:31:10.752646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:10.754291 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:10.781479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fd4f1-6f55-477a-8534-3b37b4aa01f0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:10.792005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:10.792549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:10.792948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:10.793081 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] PATH_INFO: `/v3/projects/1883384b97484ce1b0b1d9dc58df9930` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:10.794322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1883384b97484ce1b0b1d9dc58df9930)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:10.796530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:10.884167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1883384b97484ce1b0b1d9dc58df9930', 'request_id': 'req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:10.894275 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1883384b97484ce1b0b1d9dc58df9930', 'request_id': 'req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:10.904776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1883384b97484ce1b0b1d9dc58df9930', 'request_id': 'req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:10.919227 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] The token cache is being invalidate because project 1883384b97484ce1b0b1d9dc58df9930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:31:10.919813 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:10.920537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-09a8b47e-f00d-44c4-bdd3-eeecbd2a0153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:10.922118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1130/2268] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:31:10 2022] DELETE /identity/v3/projects/1883384b97484ce1b0b1d9dc58df9930 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:31:10.927243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041cce5-0dc0-4222-8be2-30f1aef6c790 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:10.927723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041cce5-0dc0-4222-8be2-30f1aef6c790 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:10.927927 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041cce5-0dc0-4222-8be2-30f1aef6c790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:10.951379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-a041cce5-0dc0-4222-8be2-30f1aef6c790 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:10.952616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a041cce5-0dc0-4222-8be2-30f1aef6c790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:10.973933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:11.027430 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1139/2269] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:11.036545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee97272-0b35-40bb-9642-708c58b79210 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.036830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee97272-0b35-40bb-9642-708c58b79210 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.037128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee97272-0b35-40bb-9642-708c58b79210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.058054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4ee97272-0b35-40bb-9642-708c58b79210 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:11.059865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4ee97272-0b35-40bb-9642-708c58b79210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:11.068180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1131/2270] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:11.072683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041cce5-0dc0-4222-8be2-30f1aef6c790 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.082846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041cce5-0dc0-4222-8be2-30f1aef6c790 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.087779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.088520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.088861 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.089226 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.089944 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:31:11.090312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.092264 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.117731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1140/2271] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:31:11.124120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee97272-0b35-40bb-9642-708c58b79210 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.134846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee97272-0b35-40bb-9642-708c58b79210 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.140369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.141303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.141802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.142278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.143128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.145262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.171759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:11.179882 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1132/2272] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:31:11 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:31:11.184980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.195239 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5910c2-98ec-4617-8c71-6666932de175 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.260669 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.261404 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.261749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.262088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.263227 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.265581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.288863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1141/2273] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:31:11 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:31:11.293011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.303980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860e1a5-0a75-4f94-a315-15c559512266 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.309240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.309850 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.310130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.310402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.311105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.312779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.323439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1133/2274] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:31:11.327449 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.337823 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d355703-7453-4d1d-aad1-25080f13ede3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.343712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb308a-675c-4c37-a2dc-86f19db658e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.345166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb308a-675c-4c37-a2dc-86f19db658e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.346168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb308a-675c-4c37-a2dc-86f19db658e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.346533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb308a-675c-4c37-a2dc-86f19db658e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.347584 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb308a-675c-4c37-a2dc-86f19db658e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.349235 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb308a-675c-4c37-a2dc-86f19db658e9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.364396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1142/2275] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:31:11.365711 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.376509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b126c-a034-45a5-be25-3fc172dacde8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.382217 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.383085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.383491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.383845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] PATH_INFO: `/v3/projects/3473e51e3eb04c7e94cdb185a6701fa1/users/abe7cd01a9b04a13a0c6c35d424cb7f5/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.400989 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3473e51e3eb04c7e94cdb185a6701fa1, user_id=abe7cd01a9b04a13a0c6c35d424cb7f5, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.404406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.417753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1134/2276] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:31:11 2022] PUT /identity/v3/projects/3473e51e3eb04c7e94cdb185a6701fa1/users/abe7cd01a9b04a13a0c6c35d424cb7f5/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:31:11.428785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d41e1-0c70-45cf-94a0-fdbec2d2062e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.429066 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d41e1-0c70-45cf-94a0-fdbec2d2062e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.429334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d41e1-0c70-45cf-94a0-fdbec2d2062e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.457373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5a6d41e1-0c70-45cf-94a0-fdbec2d2062e None None] MFA Rules not processed for user `abe7cd01a9b04a13a0c6c35d424cb7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:11.458585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5a6d41e1-0c70-45cf-94a0-fdbec2d2062e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:11.530209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1143/2277] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:11.544331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.546720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:11.667088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82aa7d-754a-4406-83ab-36f79f47402d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.676146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.676456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.676690 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.676854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.677611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.678039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.678456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.688359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.690238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.701428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1135/2278] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:31:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:31:11.707011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:11.708971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:11.726621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:11.731903 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.794054 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1144/2279] 10.222.0.48 () {58 vars in 967 bytes} [Thu Oct 27 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:11.823607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:11.825806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:11.827384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c17c4-2c81-4690-8054-9c092797b0f9 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.832088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.832975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.833249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.833521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] PATH_INFO: `/v3/users/abe7cd01a9b04a13a0c6c35d424cb7f5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.834676 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe7cd01a9b04a13a0c6c35d424cb7f5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.836623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.892658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe7cd01a9b04a13a0c6c35d424cb7f5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:11.905537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe7cd01a9b04a13a0c6c35d424cb7f5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:11.906392 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe7cd01a9b04a13a0c6c35d424cb7f5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:11.911305 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:11.917021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe7cd01a9b04a13a0c6c35d424cb7f5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:11.917591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a211b-490d-447a-9846-218c8ae35112 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:11.924981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:11.925773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.926114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.926541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.928380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-7b93173d-d88d-4d68-aa92-1ae0222508f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe7cd01a9b04a13a0c6c35d424cb7f5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:11.936321 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1136/2280] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:31:11 2022] DELETE /identity/v3/users/abe7cd01a9b04a13a0c6c35d424cb7f5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:31:11.939294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:11.940769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:11.944306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:11.944594 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:11.944859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:11.951198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1145/2281] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:31:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 27 13:31:11.970771 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:11.972086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:12.084279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1137/2282] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:12.099263 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:12.101894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:12.200070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf39776-9eff-4ada-970e-dd58201fc010 service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:12.287179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ab16b-4571-4a2a-8f6e-2a88f726a0a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:12.287752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ab16b-4571-4a2a-8f6e-2a88f726a0a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:12.288139 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ab16b-4571-4a2a-8f6e-2a88f726a0a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:12.288368 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ab16b-4571-4a2a-8f6e-2a88f726a0a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:12.300314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ab16b-4571-4a2a-8f6e-2a88f726a0a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:12.301761 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ab16b-4571-4a2a-8f6e-2a88f726a0a4 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:12.314389 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1146/2283] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:31:12.605625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:12.607259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:12.642560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77445ad9-0ba9-4970-b73f-646d9116159f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:12.654328 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:12.654704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:12.655051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:12.657260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:12.657897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:12.658171 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:12.658445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] PATH_INFO: `/v3/projects/3473e51e3eb04c7e94cdb185a6701fa1` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:12.659555 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3473e51e3eb04c7e94cdb185a6701fa1)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:12.661649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:12.683698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] MFA Rules not processed for user `261dfac380ce4eb28eb197cd512b6edc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:12.684473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:12.689985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:12.744413 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3473e51e3eb04c7e94cdb185a6701fa1', 'request_id': 'req-b0620d52-ac69-4913-8686-f5cbea13fbbb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:12.760398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3473e51e3eb04c7e94cdb185a6701fa1', 'request_id': 'req-b0620d52-ac69-4913-8686-f5cbea13fbbb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:12.763799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1147/2284] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:31:12.769440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3473e51e3eb04c7e94cdb185a6701fa1', 'request_id': 'req-b0620d52-ac69-4913-8686-f5cbea13fbbb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:12.781350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] The token cache is being invalidate because project 3473e51e3eb04c7e94cdb185a6701fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:31:12.781685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:12.782354 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b0620d52-ac69-4913-8686-f5cbea13fbbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:12.784828 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1138/2285] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:31:12 2022] DELETE /identity/v3/projects/3473e51e3eb04c7e94cdb185a6701fa1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:31:22.751041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=20 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:22.777891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:24.566486 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:24.567905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:24.580169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a85ac-cdfa-4352-8c7c-885e9338ce38 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:24.637221 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:24.637830 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:24.638109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:24.638379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:24.640104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:24.701316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:24.702731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:24.712293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1148/2286] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:31:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:31:34.707848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=50 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:34.963674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c93fdb-4a34-4f99-94cd-c42327319815 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:34.964383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c93fdb-4a34-4f99-94cd-c42327319815 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:34.964383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c93fdb-4a34-4f99-94cd-c42327319815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:34.983795 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-f2c93fdb-4a34-4f99-94cd-c42327319815 None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:34.985021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f2c93fdb-4a34-4f99-94cd-c42327319815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:35.039511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1139/2287] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Oct 27 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:31:35.052009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:35.053472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:35.123577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9568fb6-124b-422e-ad6d-eca59bf3374a service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:35.128606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:35.129211 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:35.129494 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:35.129764 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:35.137933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:35.139500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:35.147416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1149/2288] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:31:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:31:45.035713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=25 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:45.144410 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=32 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:50.572889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:50.573355 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:50.573482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:50.593613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] MFA Rules not processed for user `bc93e3b174024520ac01244301f179bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:31:50.594809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:50.602109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1140/2289] 10.222.0.48 () {58 vars in 967 bytes} [Thu Oct 27 13:31:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:31:50.615230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:50.626241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d30d-8f55-4b3b-af48-4ddb8d3a75da service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:50.631433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb1a2c4a51f46c1be39baf0927eff4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:50.632045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:50.632343 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:50.632645 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:50.641721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:50.643292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:50.652039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1150/2290] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Oct 27 13:31:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 27 13:31:52.599675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:52.601523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:52.687036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7454024-5502-40a6-8564-c49edd73fcbf None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:52.692091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:52.692266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:52.692518 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:52.692791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] PATH_INFO: `/v3/users/261dfac380ce4eb28eb197cd512b6edc` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:52.693941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=261dfac380ce4eb28eb197cd512b6edc)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:52.695863 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:52.747252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261dfac380ce4eb28eb197cd512b6edc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:52.762704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261dfac380ce4eb28eb197cd512b6edc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:52.763432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261dfac380ce4eb28eb197cd512b6edc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:52.770709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261dfac380ce4eb28eb197cd512b6edc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:52.779480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261dfac380ce4eb28eb197cd512b6edc'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:52.786021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1141/2291] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:31:52 2022] DELETE /identity/v3/users/261dfac380ce4eb28eb197cd512b6edc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:31:52.965738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:52.977857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaff9ef-9bb9-4521-a6fa-8c6f33f4fa4e service swift] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:53.035893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:53.036574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:53.036828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:53.037308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] PATH_INFO: `/v3/projects/9f309e5e9bf84fff939c088e7c256bfb` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:53.038416 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f309e5e9bf84fff939c088e7c256bfb)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:53.040344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:53.086857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f309e5e9bf84fff939c088e7c256bfb', 'request_id': 'req-a6100e2b-cc64-426a-82d7-e958df5c7305'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:53.094627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f309e5e9bf84fff939c088e7c256bfb', 'request_id': 'req-a6100e2b-cc64-426a-82d7-e958df5c7305'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:53.110731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f309e5e9bf84fff939c088e7c256bfb', 'request_id': 'req-a6100e2b-cc64-426a-82d7-e958df5c7305'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:53.122480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] The token cache is being invalidate because project 9f309e5e9bf84fff939c088e7c256bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:31:53.122766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:53.123422 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:31:53.124962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1151/2292] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:31:52 2022] DELETE /identity/v3/projects/9f309e5e9bf84fff939c088e7c256bfb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:31:59.211706 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:31:59.213298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:59.297992 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:31:59.305667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd429-90c6-40a1-8201-42cc53a1ba7f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:31:59.360710 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:31:59.361314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:31:59.361587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:31:59.361856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:31:59.363666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:31:59.473126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:31:59.474559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:31:59.495581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1142/2293] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:02.656197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:02.659779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:02.730652 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:02.732861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:02.782785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8fe62-cbad-457e-b3ad-fbc794cc4e0b service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:02.797859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:02.798845 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:02.799264 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:02.799684 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] PATH_INFO: `/v3/users/e77228bb97fd40dfa311088f28864637` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:02.801587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77228bb97fd40dfa311088f28864637)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:02.809635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:02.829633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:02.879518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6100e2b-cc64-426a-82d7-e958df5c7305 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:02.885551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:02.886294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:02.886588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:02.886857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:02.888485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77228bb97fd40dfa311088f28864637'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:02.888629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:02.895778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:02.901293 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:02.901768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77228bb97fd40dfa311088f28864637'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:02.902528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77228bb97fd40dfa311088f28864637'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:02.903114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:02.909778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77228bb97fd40dfa311088f28864637'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:02.913164 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1152/2294] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:02.918948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e29f0be7-cc84-4cbc-a12a-7380097a597c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77228bb97fd40dfa311088f28864637'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:02.932720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1143/2295] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:02 2022] DELETE /identity/v3/users/e77228bb97fd40dfa311088f28864637 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:03.102080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.113629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caefea-3630-441d-831c-5cf8073e5b39 service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.176702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.177336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.177615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.177889 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] PATH_INFO: `/v3/projects/a14a3cf9df9f4034a8a0109fd3e33816` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.179102 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14a3cf9df9f4034a8a0109fd3e33816)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.181295 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.234109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14a3cf9df9f4034a8a0109fd3e33816', 'request_id': 'req-df80e066-a834-4742-94ec-63c2e65e808e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:03.241223 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14a3cf9df9f4034a8a0109fd3e33816', 'request_id': 'req-df80e066-a834-4742-94ec-63c2e65e808e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:03.247693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14a3cf9df9f4034a8a0109fd3e33816', 'request_id': 'req-df80e066-a834-4742-94ec-63c2e65e808e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:03.260502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] The token cache is being invalidate because project a14a3cf9df9f4034a8a0109fd3e33816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:03.260787 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:03.261448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-df80e066-a834-4742-94ec-63c2e65e808e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:03.262897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1153/2296] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:03 2022] DELETE /identity/v3/projects/a14a3cf9df9f4034a8a0109fd3e33816 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:03.267799 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de8f37-4e15-4db1-aa1c-505dbfb349f9 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.268082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de8f37-4e15-4db1-aa1c-505dbfb349f9 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.268403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de8f37-4e15-4db1-aa1c-505dbfb349f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.328150 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-06de8f37-4e15-4db1-aa1c-505dbfb349f9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:03.329409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-06de8f37-4e15-4db1-aa1c-505dbfb349f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:03.392363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1144/2297] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:03.400618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ee995-6bba-40e9-abd3-974b81ebd32d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.400893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ee995-6bba-40e9-abd3-974b81ebd32d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.401155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ee995-6bba-40e9-abd3-974b81ebd32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.420697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-860ee995-6bba-40e9-abd3-974b81ebd32d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:03.421911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-860ee995-6bba-40e9-abd3-974b81ebd32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:03.430245 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1154/2298] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:03.434618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de8f37-4e15-4db1-aa1c-505dbfb349f9 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.444063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de8f37-4e15-4db1-aa1c-505dbfb349f9 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.449156 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.449743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.450015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.450280 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.450910 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:03.451205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.452816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.464841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1145/2299] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:03.469030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ee995-6bba-40e9-abd3-974b81ebd32d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.478050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ee995-6bba-40e9-abd3-974b81ebd32d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.482615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.483195 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.483469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.483732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.484378 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.486224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.515810 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1155/2300] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:03 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:03.519986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.530693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d048fb-ec19-401a-8b16-89dbbedf74c8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.587173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.587758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.588032 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.588332 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.588968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.590870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.615526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1146/2301] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:03 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:03.625365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.636654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44828a4a-f3ce-44f6-9730-5f686b63834b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.641827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.642436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.642803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.643074 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.643712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.645720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.656987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1156/2302] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:03.661124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.671185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4aea4-7ca8-4e99-9543-c64174ac754e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.675920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8a148-e5fa-41ba-a14c-696e86515f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.676578 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8a148-e5fa-41ba-a14c-696e86515f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.676833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8a148-e5fa-41ba-a14c-696e86515f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.677106 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8a148-e5fa-41ba-a14c-696e86515f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.677744 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8a148-e5fa-41ba-a14c-696e86515f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.679331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8a148-e5fa-41ba-a14c-696e86515f67 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.690044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1147/2303] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:03.694191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.704727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb99fe5-f160-4d9d-af74-ea4f54bc47a5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.709323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.709965 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.710241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.710514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] PATH_INFO: `/v3/projects/22fe10c508cc4098b12a7b996f3eb771/users/87a2da6d5c4a438b829eab16e9b65361/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.721945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:03.726795 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22fe10c508cc4098b12a7b996f3eb771, user_id=87a2da6d5c4a438b829eab16e9b65361, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.729908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.742565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1157/2304] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:03 2022] PUT /identity/v3/projects/22fe10c508cc4098b12a7b996f3eb771/users/87a2da6d5c4a438b829eab16e9b65361/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:03.754890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d7e0-ebb7-4da7-aded-c32628689b2d None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.755391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d7e0-ebb7-4da7-aded-c32628689b2d None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.755513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d7e0-ebb7-4da7-aded-c32628689b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.775158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:03.785198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-69c7d7e0-ebb7-4da7-aded-c32628689b2d None None] MFA Rules not processed for user `87a2da6d5c4a438b829eab16e9b65361`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:03.786560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-69c7d7e0-ebb7-4da7-aded-c32628689b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:03.849805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1148/2305] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:03.861243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:03.862934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:03.963253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442cd52-ab2d-4d6d-8dbb-925904c14c48 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:03.968863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b044c7e-0a8a-4d57-8560-fa8c63febd5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:03.969468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b044c7e-0a8a-4d57-8560-fa8c63febd5e service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:03.969746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b044c7e-0a8a-4d57-8560-fa8c63febd5e service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:03.970017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b044c7e-0a8a-4d57-8560-fa8c63febd5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:03.977684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b044c7e-0a8a-4d57-8560-fa8c63febd5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:03.979252 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b044c7e-0a8a-4d57-8560-fa8c63febd5e service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:03.987974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1158/2306] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:11.396911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7d7e0-ebb7-4da7-aded-c32628689b2d None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:11.464437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7d7e0-ebb7-4da7-aded-c32628689b2d None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:11.469510 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:11.470180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:11.470548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:11.470935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] PATH_INFO: `/v3/users/87a2da6d5c4a438b829eab16e9b65361` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:11.471952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a2da6d5c4a438b829eab16e9b65361)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:11.474064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:11.543470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a2da6d5c4a438b829eab16e9b65361'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:11.556677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a2da6d5c4a438b829eab16e9b65361'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:11.557491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a2da6d5c4a438b829eab16e9b65361'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:11.565114 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a2da6d5c4a438b829eab16e9b65361'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:11.573500 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a2da6d5c4a438b829eab16e9b65361'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:11.579713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1149/2307] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:11 2022] DELETE /identity/v3/users/87a2da6d5c4a438b829eab16e9b65361 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:11.584159 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:11.584695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:11.584794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:11.605307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:11.606604 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:11.671041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1159/2308] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:11.681601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:11.682991 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:11.710880 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:11.751457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd10f30-abdb-480d-a65e-a32be07764ca admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:11.755195 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702753a3-5df0-4d42-8500-24818b932ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:11.755792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702753a3-5df0-4d42-8500-24818b932ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:11.756076 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702753a3-5df0-4d42-8500-24818b932ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:11.756365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702753a3-5df0-4d42-8500-24818b932ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:11.766051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702753a3-5df0-4d42-8500-24818b932ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:11.767409 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702753a3-5df0-4d42-8500-24818b932ef8 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:11.777477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1150/2309] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:12.043720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.045158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:12.059714 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fe297-b77a-4d44-8845-9c9d9b3b24ad None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.064334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.064970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.065251 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.066129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] PATH_INFO: `/v3/projects/22fe10c508cc4098b12a7b996f3eb771` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.067440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] RBAC: Authorizing `identity:delete_project(project_id=22fe10c508cc4098b12a7b996f3eb771)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.069399 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.112272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:12.116829 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22fe10c508cc4098b12a7b996f3eb771', 'request_id': 'req-b43fa99f-e1e1-46fd-93c9-039a876545da'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:12.125492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22fe10c508cc4098b12a7b996f3eb771', 'request_id': 'req-b43fa99f-e1e1-46fd-93c9-039a876545da'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:12.131116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22fe10c508cc4098b12a7b996f3eb771', 'request_id': 'req-b43fa99f-e1e1-46fd-93c9-039a876545da'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:12.143394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] The token cache is being invalidate because project 22fe10c508cc4098b12a7b996f3eb771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:12.143681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:12.144353 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b43fa99f-e1e1-46fd-93c9-039a876545da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:12.145824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1160/2310] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:12 2022] DELETE /identity/v3/projects/22fe10c508cc4098b12a7b996f3eb771 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:12.150651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3e08a-5297-4a5b-a180-79abeccfabb5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.150927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3e08a-5297-4a5b-a180-79abeccfabb5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.151199 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3e08a-5297-4a5b-a180-79abeccfabb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.170859 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-90f3e08a-5297-4a5b-a180-79abeccfabb5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:12.172035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90f3e08a-5297-4a5b-a180-79abeccfabb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:12.230063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1151/2311] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:12.237783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddd635-9848-42a6-8663-c92aca2a044d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.238056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddd635-9848-42a6-8663-c92aca2a044d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.238320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddd635-9848-42a6-8663-c92aca2a044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.257738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-83ddd635-9848-42a6-8663-c92aca2a044d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:12.258902 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-83ddd635-9848-42a6-8663-c92aca2a044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:12.266555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1161/2312] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:12.270751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3e08a-5297-4a5b-a180-79abeccfabb5 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.280865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3e08a-5297-4a5b-a180-79abeccfabb5 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.285623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.286211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.286537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.286760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.287390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:12.287692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.289935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.301373 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1152/2313] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:12.305648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddd635-9848-42a6-8663-c92aca2a044d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.314576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddd635-9848-42a6-8663-c92aca2a044d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.319200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.319775 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.320046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.320427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.321043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.322895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.350116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1162/2314] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:12 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:12.355090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.365051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320bb54-25a8-40fa-9fdb-41c473d9e810 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.421553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.422206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.422479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.422745 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.423361 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.425293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.446024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1153/2315] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:12 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:12.450092 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.460575 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362bc41-7e3e-4e35-870a-44e62dd862cd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.465706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.466297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.466574 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.466847 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.467493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.469094 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.480285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1163/2316] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:12.484263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.494323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f64f50-995b-49f5-a0c5-e38ae6dd8856 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.499284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e622c5b-66c4-4b31-8399-89bd28aac54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.499873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e622c5b-66c4-4b31-8399-89bd28aac54f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.500145 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e622c5b-66c4-4b31-8399-89bd28aac54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.500444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e622c5b-66c4-4b31-8399-89bd28aac54f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.501069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e622c5b-66c4-4b31-8399-89bd28aac54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.502631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e622c5b-66c4-4b31-8399-89bd28aac54f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.513764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1154/2317] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:12.517866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.528469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c768744-a7f1-403a-b746-88dc8b0479a9 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.532994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.533634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.533904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.534168 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] PATH_INFO: `/v3/projects/e541f90748134fb1af00d1d46e573596/users/1b21369c3c744a039e563e5914437c42/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.548393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e541f90748134fb1af00d1d46e573596, user_id=1b21369c3c744a039e563e5914437c42, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.550934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.563534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1164/2318] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:12 2022] PUT /identity/v3/projects/e541f90748134fb1af00d1d46e573596/users/1b21369c3c744a039e563e5914437c42/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:12.574564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27735-a8b4-498a-abca-ffe434fb6bda None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.574896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27735-a8b4-498a-abca-ffe434fb6bda None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.575174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27735-a8b4-498a-abca-ffe434fb6bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.604965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-49f27735-a8b4-498a-abca-ffe434fb6bda None None] MFA Rules not processed for user `1b21369c3c744a039e563e5914437c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:12.610287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-49f27735-a8b4-498a-abca-ffe434fb6bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:12.626862 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:12.680305 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1155/2319] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:12.684819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.751382 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed66ad-2c71-431d-8c19-28e60ba12cf0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.757025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.757678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.758085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.761030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.761030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.761030 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.791920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1165/2320] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:12 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:12.796489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f27735-a8b4-498a-abca-ffe434fb6bda None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.808163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f27735-a8b4-498a-abca-ffe434fb6bda None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.875724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.876461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.877162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.877162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.877981 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.879740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.902888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1156/2321] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:12 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:12.907369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.919046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe6a23-d1f0-47eb-8f55-6724f6a59151 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.924000 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.924684 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.924961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.925233 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.925934 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.927642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.938751 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1166/2322] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:12.943198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:12.954113 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d028-755c-48ff-965f-e6c8e1382f82 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:12.959488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:12.960214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:12.960585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:12.961384 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] PATH_INFO: `/v3/projects/e63d5df29a4549f7bae7986bb210fc37/users/5c0b2107ec9946c986f06ec6339475ed/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:12.978948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63d5df29a4549f7bae7986bb210fc37, user_id=5c0b2107ec9946c986f06ec6339475ed, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:12.982062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:12.997916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1157/2323] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:12 2022] PUT /identity/v3/projects/e63d5df29a4549f7bae7986bb210fc37/users/5c0b2107ec9946c986f06ec6339475ed/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:13.002379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:13.014206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871dfed-51f6-43a8-a407-dd414f703968 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:13.055120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:13.074589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595e225-1529-414f-a497-c031f80172d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:13.075196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595e225-1529-414f-a497-c031f80172d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:13.075476 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595e225-1529-414f-a497-c031f80172d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:13.075747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595e225-1529-414f-a497-c031f80172d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:13.076421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595e225-1529-414f-a497-c031f80172d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:13.078040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595e225-1529-414f-a497-c031f80172d1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:13.088701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1167/2324] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:13.092856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:13.104175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d5dc9-4684-4a58-9694-47bc01e6a97e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:13.108838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:13.109491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:13.109770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:13.110040 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] PATH_INFO: `/v3/projects/e63d5df29a4549f7bae7986bb210fc37/users/5c0b2107ec9946c986f06ec6339475ed/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:13.111959 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63d5df29a4549f7bae7986bb210fc37, user_id=5c0b2107ec9946c986f06ec6339475ed, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:13.114628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:13.125397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1158/2325] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:13 2022] PUT /identity/v3/projects/e63d5df29a4549f7bae7986bb210fc37/users/5c0b2107ec9946c986f06ec6339475ed/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:13.137482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3380ec-fe15-49f9-b8e0-96bca80d843a None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:13.137777 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3380ec-fe15-49f9-b8e0-96bca80d843a None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:13.138051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3380ec-fe15-49f9-b8e0-96bca80d843a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:13.167700 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-bc3380ec-fe15-49f9-b8e0-96bca80d843a None None] MFA Rules not processed for user `5c0b2107ec9946c986f06ec6339475ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:13.168942 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-bc3380ec-fe15-49f9-b8e0-96bca80d843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:13.247017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1168/2326] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:13.258692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:13.260633 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:13.323216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3064a8-6250-45d6-993c-fb4045b24617 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:13.326698 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:13.327302 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:13.327574 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:13.327840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:13.337068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:13.338507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:13.347884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1159/2327] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:13.426192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3380ec-fe15-49f9-b8e0-96bca80d843a None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:13.436273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3380ec-fe15-49f9-b8e0-96bca80d843a None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:13.439970 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:13.440608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:13.440886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:13.441154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:13.498610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:13.500060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:13.510412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1169/2328] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:14.066041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:14.067722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:14.082503 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:14.147749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0968c9-a502-4bb5-9854-81931f84fe7b service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:14.156966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312e8d3-5af1-4c93-ba54-f587b9a97f5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:14.157627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312e8d3-5af1-4c93-ba54-f587b9a97f5a service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:14.157902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312e8d3-5af1-4c93-ba54-f587b9a97f5a service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:14.158168 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312e8d3-5af1-4c93-ba54-f587b9a97f5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:14.159955 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f312e8d3-5af1-4c93-ba54-f587b9a97f5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:14.173140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312e8d3-5af1-4c93-ba54-f587b9a97f5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:14.174757 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312e8d3-5af1-4c93-ba54-f587b9a97f5a service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:14.185157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1160/2329] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:16.281129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:16.342886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8c949-c6f0-41e9-8eec-5bd3eabcd55d service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:16.347436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:16.348062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:16.348392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:16.348665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] PATH_INFO: `/v3/users/1b21369c3c744a039e563e5914437c42` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:16.349773 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b21369c3c744a039e563e5914437c42)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:16.352247 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:16.354528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:16.399864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b21369c3c744a039e563e5914437c42'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:16.412063 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b21369c3c744a039e563e5914437c42'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:16.412762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b21369c3c744a039e563e5914437c42'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:16.420176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b21369c3c744a039e563e5914437c42'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:16.427555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b21369c3c744a039e563e5914437c42'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:16.434288 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1170/2330] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:16 2022] DELETE /identity/v3/users/1b21369c3c744a039e563e5914437c42 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:16.438940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:16.439397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:16.439517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:16.458020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:16.459175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:16.519230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1161/2331] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:16.529500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:16.530880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:16.609699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475e055-df62-4f5e-975a-81838f837cf4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:16.613444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:16.614046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:16.614320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:16.614585 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:16.624373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:16.625782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:16.635061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1171/2332] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:16.909146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:16.911276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:16.923878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000c73-5d3b-4ce0-8254-e4db6b104a42 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:16.928778 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:16.929447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:16.929719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:16.929985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] PATH_INFO: `/v3/projects/e541f90748134fb1af00d1d46e573596` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:16.931109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e541f90748134fb1af00d1d46e573596)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:16.933079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:16.987174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e541f90748134fb1af00d1d46e573596', 'request_id': 'req-df1a12ce-7879-451a-9887-3053d97dd220'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:16.994674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e541f90748134fb1af00d1d46e573596', 'request_id': 'req-df1a12ce-7879-451a-9887-3053d97dd220'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.001449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e541f90748134fb1af00d1d46e573596', 'request_id': 'req-df1a12ce-7879-451a-9887-3053d97dd220'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.017927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] The token cache is being invalidate because project e541f90748134fb1af00d1d46e573596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:17.018267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.018931 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.020703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1162/2333] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:16 2022] DELETE /identity/v3/projects/e541f90748134fb1af00d1d46e573596 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:17.024732 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.034406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2fec4-2bbe-48ad-a3db-1f783eae59f4 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.093772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:17.094411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.094711 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.095088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] PATH_INFO: `/v3/users/5c0b2107ec9946c986f06ec6339475ed` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.096258 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c0b2107ec9946c986f06ec6339475ed)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:17.098170 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:17.104071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:17.148542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c0b2107ec9946c986f06ec6339475ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.162057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c0b2107ec9946c986f06ec6339475ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.162771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c0b2107ec9946c986f06ec6339475ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.170327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c0b2107ec9946c986f06ec6339475ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.178772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-05208e8d-6eb1-4baf-a274-d3438e582c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c0b2107ec9946c986f06ec6339475ed'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.186602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1172/2334] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:17 2022] DELETE /identity/v3/users/5c0b2107ec9946c986f06ec6339475ed => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:17.453152 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.454702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:17.469569 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a12ce-7879-451a-9887-3053d97dd220 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.491952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:17.530785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:17.531411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.531685 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.531954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] PATH_INFO: `/v3/projects/e63d5df29a4549f7bae7986bb210fc37` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.533256 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e63d5df29a4549f7bae7986bb210fc37)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:17.535085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:17.581849 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e63d5df29a4549f7bae7986bb210fc37', 'request_id': 'req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.589790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e63d5df29a4549f7bae7986bb210fc37', 'request_id': 'req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.596027 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e63d5df29a4549f7bae7986bb210fc37', 'request_id': 'req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.608047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] The token cache is being invalidate because project e63d5df29a4549f7bae7986bb210fc37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:17.608367 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.609013 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9e29fd7c-a0b7-4c0e-9dba-d982f0aadebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:17.610480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1163/2335] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:17 2022] DELETE /identity/v3/projects/e63d5df29a4549f7bae7986bb210fc37 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:17.615529 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179667-335f-49f9-92fd-13c1a3fcda84 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.616025 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179667-335f-49f9-92fd-13c1a3fcda84 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.616144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179667-335f-49f9-92fd-13c1a3fcda84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.636166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-62179667-335f-49f9-92fd-13c1a3fcda84 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:17.637398 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-62179667-335f-49f9-92fd-13c1a3fcda84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:17.695842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1173/2336] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:17.703707 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c39e81-555b-4277-bdec-2e93021172a6 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.704012 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c39e81-555b-4277-bdec-2e93021172a6 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.704314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c39e81-555b-4277-bdec-2e93021172a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.724251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-82c39e81-555b-4277-bdec-2e93021172a6 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:17.725427 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-82c39e81-555b-4277-bdec-2e93021172a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:17.733173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1164/2337] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:17.737522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179667-335f-49f9-92fd-13c1a3fcda84 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.747699 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179667-335f-49f9-92fd-13c1a3fcda84 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.752203 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:17.752832 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.753106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.753372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.754006 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:17.754303 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:17.755866 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:17.768041 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1174/2338] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:17.772479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c39e81-555b-4277-bdec-2e93021172a6 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.782287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c39e81-555b-4277-bdec-2e93021172a6 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.787092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:17.787621 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.787894 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.788160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.788847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:17.790708 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:17.817107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1165/2339] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:17 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:17.821797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.831824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b7577-ea89-42e6-9f29-0c5e9b98563c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.887692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:17.888336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.888615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.888898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.889519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:17.891406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:17.907466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:17.916495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1175/2340] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:17 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:17.917567 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.927738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed35e5-9389-4046-baaa-e38e9a572d90 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.933011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:17.933595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.933867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.934134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.934760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:17.936363 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:17.946117 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1166/2341] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:17.950116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.960923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dca01-508f-42da-b995-577cafbe72c8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.965535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d0cfb-73fb-4ee5-a114-c35562eb92b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:17.966134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d0cfb-73fb-4ee5-a114-c35562eb92b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:17.966411 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d0cfb-73fb-4ee5-a114-c35562eb92b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:17.966707 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d0cfb-73fb-4ee5-a114-c35562eb92b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:17.967444 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d0cfb-73fb-4ee5-a114-c35562eb92b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:17.969073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d0cfb-73fb-4ee5-a114-c35562eb92b9 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:17.980300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1176/2342] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:17.984483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:17.994914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a47372-59c8-491d-b5f1-ba5c3c7843ff admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:17.999682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.000341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.000617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.000916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] PATH_INFO: `/v3/projects/4fc3c2a1222e4e2e814784a229ffc372/users/908ea6a18f5f410593430a82b1fd48bf/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.016093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc3c2a1222e4e2e814784a229ffc372, user_id=908ea6a18f5f410593430a82b1fd48bf, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.018941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.034370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1167/2343] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:17 2022] PUT /identity/v3/projects/4fc3c2a1222e4e2e814784a229ffc372/users/908ea6a18f5f410593430a82b1fd48bf/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:18.046313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de862c-b5ba-431d-831e-943452549b74 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.046666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de862c-b5ba-431d-831e-943452549b74 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.047002 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de862c-b5ba-431d-831e-943452549b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.076209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f4de862c-b5ba-431d-831e-943452549b74 None None] MFA Rules not processed for user `908ea6a18f5f410593430a82b1fd48bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:18.077454 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f4de862c-b5ba-431d-831e-943452549b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:18.164202 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1177/2344] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:18.168288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:18.224475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:18.239272 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a3b1c-4a76-4721-b921-2d34e7b34a55 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:18.248704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.249542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.249796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.250072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.250743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.252700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.281137 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1168/2345] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:18 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:18.285491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de862c-b5ba-431d-831e-943452549b74 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:18.303595 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de862c-b5ba-431d-831e-943452549b74 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:18.360142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.361076 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.361350 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.361618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.362234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.364154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.388213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1178/2346] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:18 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:18.392529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:18.404406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbf9a6-4fd9-48de-98d1-51145595dcc0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:18.409181 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.409783 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.410061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.410333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.410969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.412599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.424654 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1169/2347] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:18.428806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:18.439152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f261-77cc-4568-b2e7-a3b9250bdc24 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:18.443922 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.445182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.445380 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.445649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] PATH_INFO: `/v3/projects/989a0e22c1564c4f868c939954378497/users/6bee5fd2ada64a36b1c774771f232ec0/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.461874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989a0e22c1564c4f868c939954378497, user_id=6bee5fd2ada64a36b1c774771f232ec0, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.464649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.476624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1179/2348] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:18 2022] PUT /identity/v3/projects/989a0e22c1564c4f868c939954378497/users/6bee5fd2ada64a36b1c774771f232ec0/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:18.480570 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:18.490486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d2f04-4342-4e21-9552-18ed0de06be5 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:18.546049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88ce87-f0ff-4412-90fa-56e3858af2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.546648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88ce87-f0ff-4412-90fa-56e3858af2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.546924 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88ce87-f0ff-4412-90fa-56e3858af2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.547194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88ce87-f0ff-4412-90fa-56e3858af2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.547819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88ce87-f0ff-4412-90fa-56e3858af2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.549460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88ce87-f0ff-4412-90fa-56e3858af2e9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.559071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1170/2349] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:18.563138 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:18.573925 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8de887-457b-4ee9-886c-1fc9c11a7188 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:18.578709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.579362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.579636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.579904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] PATH_INFO: `/v3/projects/989a0e22c1564c4f868c939954378497/users/6bee5fd2ada64a36b1c774771f232ec0/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.581921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989a0e22c1564c4f868c939954378497, user_id=6bee5fd2ada64a36b1c774771f232ec0, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.584590 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.595973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1180/2350] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:18 2022] PUT /identity/v3/projects/989a0e22c1564c4f868c939954378497/users/6bee5fd2ada64a36b1c774771f232ec0/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:18.607176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.607459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.607733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.636611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] MFA Rules not processed for user `6bee5fd2ada64a36b1c774771f232ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:18.637807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:18.704390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1171/2351] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:18.716055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:18.717841 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:18.751690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:18.776284 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b808cdf-d593-47ec-81c3-079e4fb6c678 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:18.779929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:18.780564 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:18.780838 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:18.781104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:18.789757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:18.791153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:18.800344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1181/2352] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:19.415803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:19.418001 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:19.498752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:19.511644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2716c9-7b42-4f8a-9687-89e6fc704828 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:19.511644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6a5-51b4-4ebf-9955-c7a2a0557236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:19.511644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6a5-51b4-4ebf-9955-c7a2a0557236 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:19.511644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6a5-51b4-4ebf-9955-c7a2a0557236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:19.512003 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6a5-51b4-4ebf-9955-c7a2a0557236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:19.513485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-495dc6a5-51b4-4ebf-9955-c7a2a0557236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:19.584443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6a5-51b4-4ebf-9955-c7a2a0557236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:19.586462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6a5-51b4-4ebf-9955-c7a2a0557236 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:19.596978 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1172/2353] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:20.229436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:20.240005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b040b-562a-4211-a955-51a9b571abb2 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:20.246322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:20.247186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:20.247815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:20.248262 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] PATH_INFO: `/v3/users/908ea6a18f5f410593430a82b1fd48bf` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:20.250145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=908ea6a18f5f410593430a82b1fd48bf)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:20.252962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:20.310452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908ea6a18f5f410593430a82b1fd48bf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.324757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908ea6a18f5f410593430a82b1fd48bf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.325508 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908ea6a18f5f410593430a82b1fd48bf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.333370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908ea6a18f5f410593430a82b1fd48bf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.341241 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908ea6a18f5f410593430a82b1fd48bf'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.347756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1182/2354] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:20 2022] DELETE /identity/v3/users/908ea6a18f5f410593430a82b1fd48bf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:20.352448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:20.352731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:20.353029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:20.373030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:20.374247 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:20.436094 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1173/2355] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:20.446900 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:20.448339 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:20.511966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa3f89-3cdb-4579-98c5-9ddd77e87b37 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:20.515856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:20.516976 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:20.517253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:20.517524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:20.527210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:20.528786 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:20.540128 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1183/2356] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:20.855087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:20.856713 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:20.870203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c11c4f-8cd7-4485-9182-2ddfd1387fa8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:20.875826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:20.876611 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:20.877121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:20.877660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] PATH_INFO: `/v3/projects/4fc3c2a1222e4e2e814784a229ffc372` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:20.879151 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc3c2a1222e4e2e814784a229ffc372)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:20.881368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:20.939458 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc3c2a1222e4e2e814784a229ffc372', 'request_id': 'req-49e4a42c-a924-48cc-8bb1-30b20eb65c86'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.949058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc3c2a1222e4e2e814784a229ffc372', 'request_id': 'req-49e4a42c-a924-48cc-8bb1-30b20eb65c86'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.956949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc3c2a1222e4e2e814784a229ffc372', 'request_id': 'req-49e4a42c-a924-48cc-8bb1-30b20eb65c86'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.973008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] The token cache is being invalidate because project 4fc3c2a1222e4e2e814784a229ffc372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:20.973731 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.975320 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:20.979837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1174/2357] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:20 2022] DELETE /identity/v3/projects/4fc3c2a1222e4e2e814784a229ffc372 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:20.984069 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:20.994692 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178da232-3ba3-4607-9239-24bbc41262cc service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:21.047022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:21.063961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:21.065330 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:21.066064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:21.066678 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] PATH_INFO: `/v3/users/6bee5fd2ada64a36b1c774771f232ec0` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:21.070113 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bee5fd2ada64a36b1c774771f232ec0)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:21.072465 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:21.134447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bee5fd2ada64a36b1c774771f232ec0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.150381 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bee5fd2ada64a36b1c774771f232ec0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.151243 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bee5fd2ada64a36b1c774771f232ec0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.169477 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bee5fd2ada64a36b1c774771f232ec0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.180621 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01c01ab2-00d6-454f-b569-07c67d67961d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bee5fd2ada64a36b1c774771f232ec0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.189578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1184/2358] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:20 2022] DELETE /identity/v3/users/6bee5fd2ada64a36b1c774771f232ec0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:21.635133 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:21.636549 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:21.650950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a42c-a924-48cc-8bb1-30b20eb65c86 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:21.678920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:21.736390 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:21.736969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:21.737263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:21.737516 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] PATH_INFO: `/v3/projects/989a0e22c1564c4f868c939954378497` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:21.738667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=989a0e22c1564c4f868c939954378497)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:21.740605 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:21.803166 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '989a0e22c1564c4f868c939954378497', 'request_id': 'req-1637532e-476a-4010-821c-1b6b4e46c823'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.813010 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '989a0e22c1564c4f868c939954378497', 'request_id': 'req-1637532e-476a-4010-821c-1b6b4e46c823'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.820988 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '989a0e22c1564c4f868c939954378497', 'request_id': 'req-1637532e-476a-4010-821c-1b6b4e46c823'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.832937 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] The token cache is being invalidate because project 989a0e22c1564c4f868c939954378497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:21.833216 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.833936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1637532e-476a-4010-821c-1b6b4e46c823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:21.835958 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1175/2359] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:21 2022] DELETE /identity/v3/projects/989a0e22c1564c4f868c939954378497 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:21.841101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3202f7d-e8f7-4e8a-b53c-3a767534b114 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:21.841600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3202f7d-e8f7-4e8a-b53c-3a767534b114 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:21.841688 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3202f7d-e8f7-4e8a-b53c-3a767534b114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:21.863071 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-f3202f7d-e8f7-4e8a-b53c-3a767534b114 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:21.864321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f3202f7d-e8f7-4e8a-b53c-3a767534b114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:21.924915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1185/2360] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:21.932742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf471793-b2c3-48da-9a71-1fc98162c51f None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:21.933240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf471793-b2c3-48da-9a71-1fc98162c51f None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:21.933360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf471793-b2c3-48da-9a71-1fc98162c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:21.953816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-bf471793-b2c3-48da-9a71-1fc98162c51f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:21.954950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-bf471793-b2c3-48da-9a71-1fc98162c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:21.963457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1176/2361] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:21.968418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3202f7d-e8f7-4e8a-b53c-3a767534b114 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:21.981321 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3202f7d-e8f7-4e8a-b53c-3a767534b114 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:21.986929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:21.987552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:21.987959 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:21.988090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:21.988774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:21.989075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:21.990797 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.007022 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1186/2362] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:22.008165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf471793-b2c3-48da-9a71-1fc98162c51f None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.021759 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf471793-b2c3-48da-9a71-1fc98162c51f None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.027483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.028101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.028519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.028796 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.029438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.031329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.070658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1177/2363] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:22 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:22.075388 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.086988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6da100-2feb-4c19-b338-a68ab8435af8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.150580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:22.159903 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.160698 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.161191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.161529 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.162232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.165642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.188876 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1187/2364] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:22 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:22.193176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.204324 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e95a0-b568-4e8d-af16-bbcfe33436bd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.213695 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.214443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.214817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.215187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.216007 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.217729 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.230000 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1178/2365] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:22.234301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.244757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa9520-792b-4a34-a76e-6f44edf23774 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.249740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900dc92-a813-4d4c-a26d-98780f7b9356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.250338 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900dc92-a813-4d4c-a26d-98780f7b9356 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.250613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900dc92-a813-4d4c-a26d-98780f7b9356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.250884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900dc92-a813-4d4c-a26d-98780f7b9356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.251517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900dc92-a813-4d4c-a26d-98780f7b9356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.253137 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900dc92-a813-4d4c-a26d-98780f7b9356 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.264391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1188/2366] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:22.268610 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.280884 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198867c-5643-4303-ba19-42028e536989 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.290752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.291476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.291755 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.292024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] PATH_INFO: `/v3/projects/88b80da3b78447609566351ba57e36ab/users/460bb4591efd4c4f9f4d511614f95283/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.307866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b80da3b78447609566351ba57e36ab, user_id=460bb4591efd4c4f9f4d511614f95283, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.310544 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.325841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1179/2367] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:22 2022] PUT /identity/v3/projects/88b80da3b78447609566351ba57e36ab/users/460bb4591efd4c4f9f4d511614f95283/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:22.338359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7c14-4ca0-4ec7-bc4e-d3964b461011 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.338989 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7c14-4ca0-4ec7-bc4e-d3964b461011 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.339333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7c14-4ca0-4ec7-bc4e-d3964b461011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.380681 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9a9c7c14-4ca0-4ec7-bc4e-d3964b461011 None None] MFA Rules not processed for user `460bb4591efd4c4f9f4d511614f95283`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:22.382504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9a9c7c14-4ca0-4ec7-bc4e-d3964b461011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:22.475393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.477730 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1189/2368] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:22.537722 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:22.558258 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c401f0-cd37-47f5-962b-516afc74a013 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.563824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.564688 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.565147 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.565571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.566420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.568997 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.622293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1180/2369] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:22 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:22.626394 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7c14-4ca0-4ec7-bc4e-d3964b461011 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.637905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7c14-4ca0-4ec7-bc4e-d3964b461011 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.705031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.705644 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.705923 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.706185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.706821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.708931 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.739495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1190/2370] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:22 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:22.743791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.755176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e9fa0-3c98-4ce3-afb9-f09ebeae0bbe admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.763404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.764078 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.764404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.764828 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.765481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.767644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.779405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1181/2371] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:22.783620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.796040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff64812-ddf7-438e-bbe8-c35c03fbbac5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.808359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.809052 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.809355 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.809629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] PATH_INFO: `/v3/projects/e6132d058d554f42b05e853d80ef5310/users/7ae9120d26dc4b1fa9f9640f6a0371d3/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.826968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6132d058d554f42b05e853d80ef5310, user_id=7ae9120d26dc4b1fa9f9640f6a0371d3, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.829779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.852438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1191/2372] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:22 2022] PUT /identity/v3/projects/e6132d058d554f42b05e853d80ef5310/users/7ae9120d26dc4b1fa9f9640f6a0371d3/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:22.857781 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.868816 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458191f0-6325-4894-8776-5a26c3ead471 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.911576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.914049 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:22.941243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.942004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.942352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.942686 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.943391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.945115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.949855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:22.956093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1182/2373] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:22.960262 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:22.970160 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d5c92-38e2-4bf0-936a-573f265e414d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.976426 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0c364-a3f5-4f23-94c2-cdaeeb3291bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:22.977073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c364-a3f5-4f23-94c2-cdaeeb3291bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:22.977350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c364-a3f5-4f23-94c2-cdaeeb3291bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:22.977648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c364-a3f5-4f23-94c2-cdaeeb3291bf admin admin] PATH_INFO: `/v3/projects/e6132d058d554f42b05e853d80ef5310/users/7ae9120d26dc4b1fa9f9640f6a0371d3/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:22.979666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0c364-a3f5-4f23-94c2-cdaeeb3291bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6132d058d554f42b05e853d80ef5310, user_id=7ae9120d26dc4b1fa9f9640f6a0371d3, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:22.982591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0c364-a3f5-4f23-94c2-cdaeeb3291bf admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:22.987334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ac71c-497f-4ddf-b088-9ff004eebc76 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:22.994782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1183/2374] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:22 2022] PUT /identity/v3/projects/e6132d058d554f42b05e853d80ef5310/users/7ae9120d26dc4b1fa9f9640f6a0371d3/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:23.005261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.005622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.006185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.033732 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] MFA Rules not processed for user `7ae9120d26dc4b1fa9f9640f6a0371d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:23.034949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.043379 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.044011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.044314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.044586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] PATH_INFO: `/v3/users/a385ba4ea6ee4a2fa76f0f12719427ee` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.045701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a385ba4ea6ee4a2fa76f0f12719427ee)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.047605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.094299 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a385ba4ea6ee4a2fa76f0f12719427ee'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.106545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1184/2375] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:23.107362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a385ba4ea6ee4a2fa76f0f12719427ee'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.108250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a385ba4ea6ee4a2fa76f0f12719427ee'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.115691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a385ba4ea6ee4a2fa76f0f12719427ee'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.118526 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.119902 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.124555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a385ba4ea6ee4a2fa76f0f12719427ee'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.130719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1192/2376] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:22 2022] DELETE /identity/v3/users/a385ba4ea6ee4a2fa76f0f12719427ee => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:23.153101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:23.201933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa502720-ac27-4e34-9a6c-1362cade198e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.252909 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.253521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.253797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.254067 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.255810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.308146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.309767 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.328508 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.330420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.342673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1185/2377] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:23.378090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:23.390104 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f5ce-b352-4329-a618-a8b832c55b04 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.392111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.393628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.395279 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.395909 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.396259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.396518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] PATH_INFO: `/v3/projects/095dae174c664eafaf3ed463d8b7f319` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.398145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095dae174c664eafaf3ed463d8b7f319)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.400021 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.405433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20262e36-2072-45f6-a0e1-8522bc3ca5ab service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.409581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81295ede-3686-441b-b712-d85143c5bafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.410174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81295ede-3686-441b-b712-d85143c5bafb service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.410448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81295ede-3686-441b-b712-d85143c5bafb service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.410740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81295ede-3686-441b-b712-d85143c5bafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.413631 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-81295ede-3686-441b-b712-d85143c5bafb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.450791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095dae174c664eafaf3ed463d8b7f319', 'request_id': 'req-351d2090-6ad2-43f3-9d51-fc71af3b3875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.458979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095dae174c664eafaf3ed463d8b7f319', 'request_id': 'req-351d2090-6ad2-43f3-9d51-fc71af3b3875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.465287 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095dae174c664eafaf3ed463d8b7f319', 'request_id': 'req-351d2090-6ad2-43f3-9d51-fc71af3b3875'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.474380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81295ede-3686-441b-b712-d85143c5bafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.476399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81295ede-3686-441b-b712-d85143c5bafb service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.477911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] The token cache is being invalidate because project 095dae174c664eafaf3ed463d8b7f319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:23.478311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.478964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-351d2090-6ad2-43f3-9d51-fc71af3b3875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:23.480439 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1193/2378] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:23 2022] DELETE /identity/v3/projects/095dae174c664eafaf3ed463d8b7f319 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:23.485152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038e057-4221-470f-aa1e-ddbca607878d None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.485436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038e057-4221-470f-aa1e-ddbca607878d None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.485709 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038e057-4221-470f-aa1e-ddbca607878d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.488951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1186/2379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:23.506362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c038e057-4221-470f-aa1e-ddbca607878d None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:23.507608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c038e057-4221-470f-aa1e-ddbca607878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.567598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1194/2380] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:23.576338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc78e07-1688-4f9e-a024-be3c6656e4c8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.576740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc78e07-1688-4f9e-a024-be3c6656e4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.577125 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc78e07-1688-4f9e-a024-be3c6656e4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.599004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-8bc78e07-1688-4f9e-a024-be3c6656e4c8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:23.600186 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-8bc78e07-1688-4f9e-a024-be3c6656e4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:23.608802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1187/2381] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:23.617183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038e057-4221-470f-aa1e-ddbca607878d None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.623157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038e057-4221-470f-aa1e-ddbca607878d None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.628607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.629218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.629490 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.629755 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.630386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:23.630679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.632910 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.645545 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1195/2382] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:23.650689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc78e07-1688-4f9e-a024-be3c6656e4c8 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.660147 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc78e07-1688-4f9e-a024-be3c6656e4c8 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.665892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.666585 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.666934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.667255 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.667967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.670035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.675178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=100 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:23.702348 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1188/2383] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:23 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:23.706772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.717974 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aadb9-451a-428d-8f52-781f25c3abdd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.803624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.804356 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.804637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.804908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.805535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.807747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.830360 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1196/2384] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:23 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:23.834824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.846014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aaad6e-ebda-4894-b146-6936c3e662df admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.851055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.851645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.851915 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.852178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.852833 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.854410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.864476 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1189/2385] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:23.869118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.874629 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:23.880620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9220207-54af-4a9f-9a57-c144e7f18db2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.885894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b5f68-e9bc-4477-95b3-1c654e13978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.886498 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5f68-e9bc-4477-95b3-1c654e13978c admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.886778 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5f68-e9bc-4477-95b3-1c654e13978c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.887097 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5f68-e9bc-4477-95b3-1c654e13978c admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.887748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b5f68-e9bc-4477-95b3-1c654e13978c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.889342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b5f68-e9bc-4477-95b3-1c654e13978c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.899602 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1197/2386] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:23.904107 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:23.915047 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf852b7-0771-493b-b00d-e513acd1518d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:23.919641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:23.920319 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.920603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.920872 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] PATH_INFO: `/v3/projects/4f491056927b46d7844843a6b9c9390d/users/6956c2ac2e1b4cf695af79d1d973cc77/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.936414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f491056927b46d7844843a6b9c9390d, user_id=6956c2ac2e1b4cf695af79d1d973cc77, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:23.939038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:23.952184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1190/2387] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:23 2022] PUT /identity/v3/projects/4f491056927b46d7844843a6b9c9390d/users/6956c2ac2e1b4cf695af79d1d973cc77/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:23.962495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:23.962796 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:23.963059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:23.990869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] MFA Rules not processed for user `6956c2ac2e1b4cf695af79d1d973cc77`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:23.992725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:24.055361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1198/2388] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:24.066839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:24.068542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:24.140182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa873234-854c-4c80-8c48-42eb1f23e5c2 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:24.145720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:24.146310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:24.146582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:24.146847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:24.155180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:24.156716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:24.166190 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1191/2389] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:26.802489 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:26.803964 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:26.851748 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:26.875485 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f642d-675f-41f7-ade7-3a749f712270 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:26.880988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:26.882169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:26.882441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:26.882705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] PATH_INFO: `/v3/users/460bb4591efd4c4f9f4d511614f95283` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:26.883801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=460bb4591efd4c4f9f4d511614f95283)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:26.885719 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:26.913793 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e076c3e-f82b-4860-a33d-31dbeac8809f service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:26.918815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:26.919441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:26.919717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:26.919985 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] PATH_INFO: `/v3/users/6956c2ac2e1b4cf695af79d1d973cc77` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:26.921126 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6956c2ac2e1b4cf695af79d1d973cc77)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:26.923029 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:26.931174 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '460bb4591efd4c4f9f4d511614f95283'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.943075 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '460bb4591efd4c4f9f4d511614f95283'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.943752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '460bb4591efd4c4f9f4d511614f95283'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.951541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '460bb4591efd4c4f9f4d511614f95283'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.958791 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88885460-02ab-4d48-9902-2fb86b84c064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '460bb4591efd4c4f9f4d511614f95283'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.961950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:26.964356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:26.966792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1199/2390] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:26 2022] DELETE /identity/v3/users/460bb4591efd4c4f9f4d511614f95283 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:26.971655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:26.972038 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:26.972672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:26.977564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6956c2ac2e1b4cf695af79d1d973cc77'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.989930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6956c2ac2e1b4cf695af79d1d973cc77'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.990629 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6956c2ac2e1b4cf695af79d1d973cc77'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:26.992662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:26.993871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:26.998817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6956c2ac2e1b4cf695af79d1d973cc77'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.007310 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-05caa8b0-6edd-4ff1-b710-e1765fa64055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6956c2ac2e1b4cf695af79d1d973cc77'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.014429 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1192/2391] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:26 2022] DELETE /identity/v3/users/6956c2ac2e1b4cf695af79d1d973cc77 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:27.018841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.019122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.019391 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.038951 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:27.040071 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.054587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1200/2392] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:27.065120 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.066503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.099471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1193/2393] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:27.109839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.111184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.124294 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eb55a-2321-40c4-b969-c946b2d6d3ce None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:27.170737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79540992-c3e8-4f5b-bf83-44bd242f6966 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:27.174748 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:27.175346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.175617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.175800 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:27.176059 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.176453 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.176733 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.177006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.186157 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:27.187276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:27.187581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:27.188752 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:27.199376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1201/2394] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:27.202344 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1194/2395] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:27.569062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.570660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.585333 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f713a-705e-4b69-94a6-82d7685d2141 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:27.593017 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:27.595812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.596217 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.596555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] PATH_INFO: `/v3/projects/4f491056927b46d7844843a6b9c9390d` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.598972 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f491056927b46d7844843a6b9c9390d)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:27.601686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:27.612054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.618381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.637930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c7783-40ad-4ef3-8b60-1b381c2cf019 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:27.643804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:27.644822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.645209 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.645700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] PATH_INFO: `/v3/projects/88b80da3b78447609566351ba57e36ab` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.646983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b80da3b78447609566351ba57e36ab)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:27.649075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:27.663948 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f491056927b46d7844843a6b9c9390d', 'request_id': 'req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.668311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:27.674130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f491056927b46d7844843a6b9c9390d', 'request_id': 'req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.685039 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f491056927b46d7844843a6b9c9390d', 'request_id': 'req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.698269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] The token cache is being invalidate because project 4f491056927b46d7844843a6b9c9390d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:27.698656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.699702 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-9ffedbfe-7f63-4bdb-a21d-6a1cc6fa1159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.701271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1202/2396] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:27 2022] DELETE /identity/v3/projects/4f491056927b46d7844843a6b9c9390d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:27.704801 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b80da3b78447609566351ba57e36ab', 'request_id': 'req-dc2ebaeb-416b-46f3-8716-ec05fba5152c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.706649 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388be60-8177-43f0-afc8-b7f2ce39a53b None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.706792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:27.707009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388be60-8177-43f0-afc8-b7f2ce39a53b None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.707377 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388be60-8177-43f0-afc8-b7f2ce39a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.714589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b80da3b78447609566351ba57e36ab', 'request_id': 'req-dc2ebaeb-416b-46f3-8716-ec05fba5152c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.723132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b80da3b78447609566351ba57e36ab', 'request_id': 'req-dc2ebaeb-416b-46f3-8716-ec05fba5152c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.733514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e388be60-8177-43f0-afc8-b7f2ce39a53b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:27.734817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e388be60-8177-43f0-afc8-b7f2ce39a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.734980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] The token cache is being invalidate because project 88b80da3b78447609566351ba57e36ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:27.735276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.735936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.737448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1195/2397] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:27 2022] DELETE /identity/v3/projects/88b80da3b78447609566351ba57e36ab => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:27.741925 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.743639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.800877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1203/2398] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:27.808671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53169fe-0830-4552-8cce-82da548a4a91 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.809273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53169fe-0830-4552-8cce-82da548a4a91 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.809273 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53169fe-0830-4552-8cce-82da548a4a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.818658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ebaeb-416b-46f3-8716-ec05fba5152c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:27.823438 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:27.824530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.824810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.825082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] PATH_INFO: `/v3/users/7ae9120d26dc4b1fa9f9640f6a0371d3` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.826378 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae9120d26dc4b1fa9f9640f6a0371d3)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:27.828308 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:27.829362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-b53169fe-0830-4552-8cce-82da548a4a91 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:27.830566 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-b53169fe-0830-4552-8cce-82da548a4a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:27.838919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1204/2399] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:27.843166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53169fe-0830-4552-8cce-82da548a4a91 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.852513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53169fe-0830-4552-8cce-82da548a4a91 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:27.858142 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:27.858880 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.859266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.859679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.860520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:27.860825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:27.862599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:27.878746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1205/2400] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:32:27.878926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae9120d26dc4b1fa9f9640f6a0371d3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.879811 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.889522 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37c3ec-4050-46d6-8369-b7d854059538 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:27.890689 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae9120d26dc4b1fa9f9640f6a0371d3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.891887 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae9120d26dc4b1fa9f9640f6a0371d3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.894531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:27.895116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:27.895390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:27.895659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:27.896384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:27.898734 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:27.899477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae9120d26dc4b1fa9f9640f6a0371d3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.907318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae9120d26dc4b1fa9f9640f6a0371d3'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:27.914224 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1196/2401] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:27 2022] DELETE /identity/v3/users/7ae9120d26dc4b1fa9f9640f6a0371d3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:27.929089 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1206/2402] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:27 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:27.933218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:27.934608 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.001639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855a6ad-259c-47a9-ae51-0b3f10390523 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.005602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:28.065132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.065948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.066300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.066634 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.067334 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.069658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.092651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1197/2403] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:27 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:28.096762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.107790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06004e-1fae-4298-b102-0172292b5b7c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.112618 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.113244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.113513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.113771 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.114505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.115985 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.127883 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1207/2404] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:28.130552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.141721 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c897007-2074-486b-8dc3-8aa2e60b0a47 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.146465 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.147794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.148130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.148485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.149184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.150821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.161376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1198/2405] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:28.165617 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.175738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7919e6d-2670-4038-b27f-09ae84efb27e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.180816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149b42f-8e4d-429c-8ca7-a622f5f53098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.181565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149b42f-8e4d-429c-8ca7-a622f5f53098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.181812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149b42f-8e4d-429c-8ca7-a622f5f53098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.182094 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149b42f-8e4d-429c-8ca7-a622f5f53098 admin admin] PATH_INFO: `/v3/projects/30e8ed96beb1431899d82e0fb2ac6fa7/users/830736fef42f47feb9ac68176f883a93/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.192697 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.195322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.199616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149b42f-8e4d-429c-8ca7-a622f5f53098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e8ed96beb1431899d82e0fb2ac6fa7, user_id=830736fef42f47feb9ac68176f883a93, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.203342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149b42f-8e4d-429c-8ca7-a622f5f53098 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.219670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f8d6-d7e0-46ab-8adc-6899c542b9c6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.223920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1208/2406] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:28 2022] PUT /identity/v3/projects/30e8ed96beb1431899d82e0fb2ac6fa7/users/830736fef42f47feb9ac68176f883a93/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:28.235627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.235872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.236274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.266561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] MFA Rules not processed for user `830736fef42f47feb9ac68176f883a93`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:28.267754 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.282460 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.283091 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.283364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.283632 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] PATH_INFO: `/v3/projects/e6132d058d554f42b05e853d80ef5310` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.284786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6132d058d554f42b05e853d80ef5310)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.287519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.325716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:28.337945 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1209/2407] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:28.338481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6132d058d554f42b05e853d80ef5310', 'request_id': 'req-dc9bbfd4-930c-4207-89b1-702ab929825b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:28.346519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6132d058d554f42b05e853d80ef5310', 'request_id': 'req-dc9bbfd4-930c-4207-89b1-702ab929825b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:28.349348 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.350697 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.352823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6132d058d554f42b05e853d80ef5310', 'request_id': 'req-dc9bbfd4-930c-4207-89b1-702ab929825b'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:28.365065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] The token cache is being invalidate because project e6132d058d554f42b05e853d80ef5310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:28.365352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:28.366051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-dc9bbfd4-930c-4207-89b1-702ab929825b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:28.367718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1199/2408] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:28 2022] DELETE /identity/v3/projects/e6132d058d554f42b05e853d80ef5310 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:28.372360 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71caab-1f68-4f92-bfa4-d112a87d00e8 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.372647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71caab-1f68-4f92-bfa4-d112a87d00e8 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.373218 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71caab-1f68-4f92-bfa4-d112a87d00e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.393617 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-2f71caab-1f68-4f92-bfa4-d112a87d00e8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:28.394839 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-2f71caab-1f68-4f92-bfa4-d112a87d00e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.404990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:28.411738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7a8cd-94df-45a5-b802-36ca9e418be2 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.455599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1200/2409] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:28.461828 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.462753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.463206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.463549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.463841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b388-ce98-482f-90e5-036d6b5356d0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.464129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b388-ce98-482f-90e5-036d6b5356d0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.464441 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b388-ce98-482f-90e5-036d6b5356d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.465578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.483751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-3dd9b388-ce98-482f-90e5-036d6b5356d0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:28.484962 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3dd9b388-ce98-482f-90e5-036d6b5356d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.493439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1201/2410] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:28.497889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b388-ce98-482f-90e5-036d6b5356d0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.507496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b388-ce98-482f-90e5-036d6b5356d0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.512198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.512815 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.513074 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.513336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.513961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:28.514253 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.515825 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.526660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.527651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1202/2411] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:32:28.528088 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.531907 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.537882 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1210/2412] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:28.543185 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f68f3-ac54-400c-9bb1-2090e4416c9d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.548116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.548821 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.549157 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.549491 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.550147 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.552015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.580767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1203/2413] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:28 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:28.585085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.596147 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de77c-fcc4-4247-ad73-acee312e0b32 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.661040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.661654 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.662127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.662289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.662950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.665051 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.676171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:28.696843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1211/2414] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:28 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:28.700946 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.711181 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fe675-945a-4d7f-b5ba-bc1f72523e8a admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.715841 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.716471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.716742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.717008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.717641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.719260 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.730236 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1204/2415] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:28.734190 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.744980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c692d6f-f85a-4709-a447-37eaa1088153 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.750037 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9c33b-fe9e-4505-b266-4e83c85bab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.750665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9c33b-fe9e-4505-b266-4e83c85bab7a admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.750940 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9c33b-fe9e-4505-b266-4e83c85bab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.751218 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9c33b-fe9e-4505-b266-4e83c85bab7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.751929 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9c33b-fe9e-4505-b266-4e83c85bab7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.753559 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9c33b-fe9e-4505-b266-4e83c85bab7a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.765219 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1212/2416] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:28.769323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:28.780663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd8da4-79e6-47e5-95cf-146c9ff7a301 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:28.785971 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:28.786638 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.787044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.787175 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] PATH_INFO: `/v3/projects/6d6d90aeb83747f4b6e4f90f68669c5c/users/429f9e16c06745f39841babdbdd041a5/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.807619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6d90aeb83747f4b6e4f90f68669c5c, user_id=429f9e16c06745f39841babdbdd041a5, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:28.810351 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:28.826235 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1205/2417] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:28 2022] PUT /identity/v3/projects/6d6d90aeb83747f4b6e4f90f68669c5c/users/429f9e16c06745f39841babdbdd041a5/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:28.837864 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dd6b2-8063-4196-b090-bb6bea76ccbf None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:28.838351 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dd6b2-8063-4196-b090-bb6bea76ccbf None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:28.838435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dd6b2-8063-4196-b090-bb6bea76ccbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:28.874391 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-cf7dd6b2-8063-4196-b090-bb6bea76ccbf None None] MFA Rules not processed for user `429f9e16c06745f39841babdbdd041a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:28.875935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-cf7dd6b2-8063-4196-b090-bb6bea76ccbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:28.944169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1213/2418] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:28.948866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.012240 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181f8da-1ad1-4244-b552-dbcd099cddcb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.017444 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.018053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.018333 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.018605 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.019584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.021219 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.048538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1206/2419] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:28 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:29.052664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dd6b2-8063-4196-b090-bb6bea76ccbf None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.063836 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dd6b2-8063-4196-b090-bb6bea76ccbf None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.122600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.123199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.123469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.123733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.124374 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.126308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.146483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1214/2420] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:29 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:29.150523 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.160736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b12ab-e8f4-4c77-8c2a-d7b3249205a1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.165916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.166591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.166874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.167147 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.167797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.169436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.180999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1207/2421] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:29.185199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.194913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c731a-38fa-4dc5-9f3e-664d1c988919 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.199403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.200038 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.200346 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.200615 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] PATH_INFO: `/v3/projects/c9b60840f67241e5badc8d3ae8da7bbe/users/1981af28ec224187acd0bba11c786155/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.215991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b60840f67241e5badc8d3ae8da7bbe, user_id=1981af28ec224187acd0bba11c786155, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.219423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.232675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1215/2422] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:29 2022] PUT /identity/v3/projects/c9b60840f67241e5badc8d3ae8da7bbe/users/1981af28ec224187acd0bba11c786155/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:29.236601 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.247560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb21dd-3f03-408f-8caa-73738a252d54 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.256416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:29.304315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684a210-a00f-4edc-be90-2c4a2d4b897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.304869 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684a210-a00f-4edc-be90-2c4a2d4b897f admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.305146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684a210-a00f-4edc-be90-2c4a2d4b897f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.305416 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684a210-a00f-4edc-be90-2c4a2d4b897f admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.306043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684a210-a00f-4edc-be90-2c4a2d4b897f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.307616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684a210-a00f-4edc-be90-2c4a2d4b897f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.317140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1208/2423] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:29.321296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.331322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d28a4-d07b-420a-85fb-700a05fa8222 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.336369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.337003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.337274 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.337540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] PATH_INFO: `/v3/projects/c9b60840f67241e5badc8d3ae8da7bbe/users/1981af28ec224187acd0bba11c786155/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.339387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b60840f67241e5badc8d3ae8da7bbe, user_id=1981af28ec224187acd0bba11c786155, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.342036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.353121 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1216/2424] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:29 2022] PUT /identity/v3/projects/c9b60840f67241e5badc8d3ae8da7bbe/users/1981af28ec224187acd0bba11c786155/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:29.363092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b208f-6493-4134-a915-71791070b64a None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.363377 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b208f-6493-4134-a915-71791070b64a None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.363651 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b208f-6493-4134-a915-71791070b64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.393680 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-151b208f-6493-4134-a915-71791070b64a None None] MFA Rules not processed for user `1981af28ec224187acd0bba11c786155`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:29.394824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-151b208f-6493-4134-a915-71791070b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:29.461916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1209/2425] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:29.473868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.476309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:29.508703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:29.539152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49de62f-4981-444b-bfb2-459860ac2081 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.542899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0a10e-eeb1-4040-94d0-4b40fd6f1238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.543530 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0a10e-eeb1-4040-94d0-4b40fd6f1238 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.543804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0a10e-eeb1-4040-94d0-4b40fd6f1238 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.544070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0a10e-eeb1-4040-94d0-4b40fd6f1238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.553083 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0a10e-eeb1-4040-94d0-4b40fd6f1238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.554547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0a10e-eeb1-4040-94d0-4b40fd6f1238 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.563687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1217/2426] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:29.632083 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b208f-6493-4134-a915-71791070b64a None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.692479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b208f-6493-4134-a915-71791070b64a None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.697475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.698098 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.698371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.698639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] PATH_INFO: `/v3/users/429f9e16c06745f39841babdbdd041a5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.699749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=429f9e16c06745f39841babdbdd041a5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.701670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.739538 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:29.748649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '429f9e16c06745f39841babdbdd041a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:29.761108 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '429f9e16c06745f39841babdbdd041a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:29.761797 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '429f9e16c06745f39841babdbdd041a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:29.768591 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '429f9e16c06745f39841babdbdd041a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:29.776701 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '429f9e16c06745f39841babdbdd041a5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:29.782559 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1210/2427] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:29 2022] DELETE /identity/v3/users/429f9e16c06745f39841babdbdd041a5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:29.787105 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.787431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.787723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.806455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:29.807648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:29.865911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1218/2428] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:29.884353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:29.885810 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:29.946535 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8da36d-369d-475f-b9a0-14d9ea9aaf8f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:29.950064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:29.950667 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:29.950942 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:29.951211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:29.961283 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:29.962699 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:29.972443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1211/2429] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:30.273980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:30.275798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:30.290798 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8dd5d-c5fa-4687-a063-dc3d727a50f8 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:30.296003 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:30.296850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:30.297127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:30.297392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] PATH_INFO: `/v3/projects/6d6d90aeb83747f4b6e4f90f68669c5c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:30.298705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6d90aeb83747f4b6e4f90f68669c5c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:30.300814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:30.348656 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6d90aeb83747f4b6e4f90f68669c5c', 'request_id': 'req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.356788 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6d90aeb83747f4b6e4f90f68669c5c', 'request_id': 'req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.363004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6d90aeb83747f4b6e4f90f68669c5c', 'request_id': 'req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.374928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] The token cache is being invalidate because project 6d6d90aeb83747f4b6e4f90f68669c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:30.375210 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.375842 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.377507 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1219/2430] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:30 2022] DELETE /identity/v3/projects/6d6d90aeb83747f4b6e4f90f68669c5c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:30.390070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:30.400917 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d8b85-d328-4fea-90a1-06114da4afd2 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:30.468792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:30.469388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:30.469728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:30.470034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] PATH_INFO: `/v3/users/1981af28ec224187acd0bba11c786155` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:30.471287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1981af28ec224187acd0bba11c786155)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:30.473548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:30.528369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1981af28ec224187acd0bba11c786155'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.537807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:30.543347 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1981af28ec224187acd0bba11c786155'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.544330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1981af28ec224187acd0bba11c786155'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.552095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1981af28ec224187acd0bba11c786155'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.559214 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-fd313e52-4a77-4d41-b350-24c8f1d26193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1981af28ec224187acd0bba11c786155'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.565060 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1212/2431] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:30 2022] DELETE /identity/v3/users/1981af28ec224187acd0bba11c786155 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:30.837534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:30.839593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:30.854616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ed3f9-6fa7-4d27-bd80-9a0dd9634d8c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:30.905109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:30.914968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:30.915603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:30.915881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:30.916151 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] PATH_INFO: `/v3/projects/c9b60840f67241e5badc8d3ae8da7bbe` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:30.917314 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b60840f67241e5badc8d3ae8da7bbe)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:30.919182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:30.966442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b60840f67241e5badc8d3ae8da7bbe', 'request_id': 'req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.974012 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b60840f67241e5badc8d3ae8da7bbe', 'request_id': 'req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.979679 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b60840f67241e5badc8d3ae8da7bbe', 'request_id': 'req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.992557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] The token cache is being invalidate because project c9b60840f67241e5badc8d3ae8da7bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:30.992848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.993498 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-8e89d482-7b6f-4b8e-882c-b7d2cfefd653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:30.994979 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1220/2432] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:30 2022] DELETE /identity/v3/projects/c9b60840f67241e5badc8d3ae8da7bbe => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:31.000207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c336d7-1750-48d3-aa7d-c12187b391a2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.000760 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c336d7-1750-48d3-aa7d-c12187b391a2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.000874 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c336d7-1750-48d3-aa7d-c12187b391a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.020900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-84c336d7-1750-48d3-aa7d-c12187b391a2 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:31.022087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-84c336d7-1750-48d3-aa7d-c12187b391a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:31.080164 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1213/2433] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:31.088116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6dd75-cb74-4714-b535-3c9bc31656cb None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.088428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6dd75-cb74-4714-b535-3c9bc31656cb None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.088694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6dd75-cb74-4714-b535-3c9bc31656cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.108367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-76e6dd75-cb74-4714-b535-3c9bc31656cb None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:31.109551 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-76e6dd75-cb74-4714-b535-3c9bc31656cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:31.117988 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1221/2434] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:31.122587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c336d7-1750-48d3-aa7d-c12187b391a2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.134541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c336d7-1750-48d3-aa7d-c12187b391a2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.139075 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.139674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.139952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.140222 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.140892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:31.141194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.142768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.154383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1214/2435] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:31.158807 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6dd75-cb74-4714-b535-3c9bc31656cb None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.168543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6dd75-cb74-4714-b535-3c9bc31656cb None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.173230 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.173820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.174096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.174365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.175033 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.176971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.205098 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1222/2436] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:31 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:31.209429 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.219414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00003916-5378-4ea5-9436-19ff141942fb admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.276383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.277030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.277311 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.277581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.278230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.280173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.307645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1215/2437] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:31 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:31.308877 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.319267 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee355c-aa8b-49ae-a8b0-1d3dc48822f3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.328005 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.328659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.328962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.329384 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.329857 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.331457 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.343433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1223/2438] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:31.347481 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.358079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f52ae-dc3d-4c91-9375-23b5977aeaa1 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.362936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259cd14-a45e-4c13-8401-d2f3baffc2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.363545 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259cd14-a45e-4c13-8401-d2f3baffc2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.363820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259cd14-a45e-4c13-8401-d2f3baffc2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.364090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259cd14-a45e-4c13-8401-d2f3baffc2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.364773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259cd14-a45e-4c13-8401-d2f3baffc2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.366994 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259cd14-a45e-4c13-8401-d2f3baffc2ff admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.378679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1216/2439] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:31.382973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.395863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106a4a-3089-4471-9b26-193c9839b2aa admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.401257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.402090 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.402376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.402648 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] PATH_INFO: `/v3/projects/6957bf53251942e3be78c8c599c46488/users/00ef01dba18d45bc89b1df0de84a6bbd/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.421036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6957bf53251942e3be78c8c599c46488, user_id=00ef01dba18d45bc89b1df0de84a6bbd, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.423774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.439434 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1224/2440] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:31 2022] PUT /identity/v3/projects/6957bf53251942e3be78c8c599c46488/users/00ef01dba18d45bc89b1df0de84a6bbd/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:31.508969 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebe76b-6cb4-4b36-9022-544cb742e214 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.509252 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebe76b-6cb4-4b36-9022-544cb742e214 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.509528 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebe76b-6cb4-4b36-9022-544cb742e214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.542725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-57ebe76b-6cb4-4b36-9022-544cb742e214 None None] MFA Rules not processed for user `00ef01dba18d45bc89b1df0de84a6bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:31.543895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-57ebe76b-6cb4-4b36-9022-544cb742e214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:31.577703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:31.623883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1217/2441] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:31.628183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.711631 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0accf50-0b0e-4d94-b250-4f5a4f730323 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.717453 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.719302 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.719633 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.719840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.720558 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.722492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.756433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1225/2442] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:31 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:31.757723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebe76b-6cb4-4b36-9022-544cb742e214 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.769176 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebe76b-6cb4-4b36-9022-544cb742e214 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.832402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.832933 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.833210 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.833479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.834115 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.836637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.860734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1218/2443] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:31 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:31.865143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.872826 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:31.876622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83fc95-1622-4af8-ad8b-77cd3799a322 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.883182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.883992 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.884471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.884805 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.885614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.887240 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.899651 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1226/2444] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:31.904034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.914405 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7530b8a-046b-4fd3-9345-486b0441e6b0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:31.919557 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:31.920395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:31.920709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:31.921034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] PATH_INFO: `/v3/projects/ff3e0f4e122b41e3bcc5cd637c970168/users/f6edb40cc3544e29b67dfc35415e100a/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:31.938198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3e0f4e122b41e3bcc5cd637c970168, user_id=f6edb40cc3544e29b67dfc35415e100a, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:31.941440 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:31.957484 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1219/2445] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:31 2022] PUT /identity/v3/projects/ff3e0f4e122b41e3bcc5cd637c970168/users/f6edb40cc3544e29b67dfc35415e100a/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:31.962107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:31.972819 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c5360-98ba-4470-b2f5-291c5b6320ea admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:32.032175 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5521b-f527-4a2f-93c6-d35bf1d2c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:32.032825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5521b-f527-4a2f-93c6-d35bf1d2c799 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:32.033111 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5521b-f527-4a2f-93c6-d35bf1d2c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:32.033376 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5521b-f527-4a2f-93c6-d35bf1d2c799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:32.034015 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5521b-f527-4a2f-93c6-d35bf1d2c799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:32.035603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5521b-f527-4a2f-93c6-d35bf1d2c799 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:32.045822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1227/2446] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:32.050220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:32.061080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d084-60d4-4c3a-9118-98ce6a93c19e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:32.065934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:32.066615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:32.066889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:32.067159 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] PATH_INFO: `/v3/projects/ff3e0f4e122b41e3bcc5cd637c970168/users/f6edb40cc3544e29b67dfc35415e100a/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:32.069758 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3e0f4e122b41e3bcc5cd637c970168, user_id=f6edb40cc3544e29b67dfc35415e100a, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:32.073183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:32.085775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1220/2447] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:32 2022] PUT /identity/v3/projects/ff3e0f4e122b41e3bcc5cd637c970168/users/f6edb40cc3544e29b67dfc35415e100a/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:32.096664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d60e7-e9f4-42ad-83e9-3da2672857ec None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:32.096954 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d60e7-e9f4-42ad-83e9-3da2672857ec None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:32.097244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d60e7-e9f4-42ad-83e9-3da2672857ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:32.128095 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-046d60e7-e9f4-42ad-83e9-3da2672857ec None None] MFA Rules not processed for user `f6edb40cc3544e29b67dfc35415e100a`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:32.129367 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-046d60e7-e9f4-42ad-83e9-3da2672857ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:32.202371 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1228/2448] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:32.214768 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:32.216548 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:32.275967 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbab51-5748-4648-9086-cba5c2fa258c admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:32.279619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:32.280223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:32.280529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:32.280804 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:32.289870 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:32.291303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:32.300738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1221/2449] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:32.384543 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d60e7-e9f4-42ad-83e9-3da2672857ec None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:32.394846 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d60e7-e9f4-42ad-83e9-3da2672857ec None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:32.398470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58707def-d627-46fb-8a25-b454e8e6ccef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:32.399057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58707def-d627-46fb-8a25-b454e8e6ccef service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:32.399327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58707def-d627-46fb-8a25-b454e8e6ccef service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:32.399593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58707def-d627-46fb-8a25-b454e8e6ccef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:32.454225 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58707def-d627-46fb-8a25-b454e8e6ccef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:32.455647 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58707def-d627-46fb-8a25-b454e8e6ccef service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:32.464400 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1229/2450] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:37.125488 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:37.127024 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:37.165787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:37.211852 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645df046-ae76-4e43-8e03-5040d9fa325e service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:37.216639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:37.217316 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.217595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.217888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] PATH_INFO: `/v3/users/830736fef42f47feb9ac68176f883a93` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.219134 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830736fef42f47feb9ac68176f883a93)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:37.221140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:37.276131 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830736fef42f47feb9ac68176f883a93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.291939 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830736fef42f47feb9ac68176f883a93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.292767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830736fef42f47feb9ac68176f883a93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.301035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830736fef42f47feb9ac68176f883a93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.309513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830736fef42f47feb9ac68176f883a93'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.317087 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1222/2451] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:37 2022] DELETE /identity/v3/users/830736fef42f47feb9ac68176f883a93 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:37.322716 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.323207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.323336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.327230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:37.328619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:37.343132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:37.344296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:37.352572 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:37.408896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7d6ef-b744-4a76-a0d4-d9e299256ca9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:37.412281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1230/2452] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:37.415086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:37.415086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.415086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.415086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] PATH_INFO: `/v3/users/00ef01dba18d45bc89b1df0de84a6bbd` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.416575 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ef01dba18d45bc89b1df0de84a6bbd)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:37.418207 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:37.422751 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:37.426091 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:37.470233 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ef01dba18d45bc89b1df0de84a6bbd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.471366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:37.485513 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ef01dba18d45bc89b1df0de84a6bbd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.486225 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ef01dba18d45bc89b1df0de84a6bbd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.489429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e702cb-9b61-4f61-8f1b-cee284185fab None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:37.493659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:37.493826 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ef01dba18d45bc89b1df0de84a6bbd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.494269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.494550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.494816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.508306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-6352044f-9243-40e3-abdb-f0003e064528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ef01dba18d45bc89b1df0de84a6bbd'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.511179 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:37.512687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:37.517359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1223/2453] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:37 2022] DELETE /identity/v3/users/00ef01dba18d45bc89b1df0de84a6bbd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:37.522724 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1231/2454] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:37.523220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.523647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.524105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.549627 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:37.551194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:37.637212 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1224/2455] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:37.646531 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:37.647908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:37.659161 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7c3c8-c164-4e2a-aa10-bf0a8858aca1 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:37.713580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14abc4-a1ce-40d4-afa7-d160ef9b3668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:37.714191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14abc4-a1ce-40d4-afa7-d160ef9b3668 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.714463 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14abc4-a1ce-40d4-afa7-d160ef9b3668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.714731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14abc4-a1ce-40d4-afa7-d160ef9b3668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.724129 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14abc4-a1ce-40d4-afa7-d160ef9b3668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:37.725586 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14abc4-a1ce-40d4-afa7-d160ef9b3668 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:37.735207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1232/2456] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:37.843030 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:37.844521 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:37.857782 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbf0fc-34ad-4cc9-8b6c-0e1cafd4ddc0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:37.862712 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:37.863368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.863644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.863947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] PATH_INFO: `/v3/projects/30e8ed96beb1431899d82e0fb2ac6fa7` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.865867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e8ed96beb1431899d82e0fb2ac6fa7)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:37.867829 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:37.931213 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e8ed96beb1431899d82e0fb2ac6fa7', 'request_id': 'req-330f91a6-040f-496c-8033-779765815c23'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.939596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e8ed96beb1431899d82e0fb2ac6fa7', 'request_id': 'req-330f91a6-040f-496c-8033-779765815c23'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.946406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e8ed96beb1431899d82e0fb2ac6fa7', 'request_id': 'req-330f91a6-040f-496c-8033-779765815c23'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.965452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] The token cache is being invalidate because project 30e8ed96beb1431899d82e0fb2ac6fa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:37.966014 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.967193 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-330f91a6-040f-496c-8033-779765815c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:37.969376 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1225/2457] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:37 2022] DELETE /identity/v3/projects/30e8ed96beb1431899d82e0fb2ac6fa7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:37.976173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:37.976577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:37.976869 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:37.998186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:37.999528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.053785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=100 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:38.063108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1233/2458] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:38.071059 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22dc8c-fb7f-4b17-8390-1778477842f7 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.071341 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22dc8c-fb7f-4b17-8390-1778477842f7 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.071615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22dc8c-fb7f-4b17-8390-1778477842f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.095807 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1f22dc8c-fb7f-4b17-8390-1778477842f7 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:38.097223 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1f22dc8c-fb7f-4b17-8390-1778477842f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.099130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.101540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.106318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1226/2459] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:38.110620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22dc8c-fb7f-4b17-8390-1778477842f7 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.116258 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7ba81-018c-4a05-9b45-9a31d998b9aa None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.120963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22dc8c-fb7f-4b17-8390-1778477842f7 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.121309 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.122040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.122428 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.122792 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] PATH_INFO: `/v3/projects/6957bf53251942e3be78c8c599c46488` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.124027 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6957bf53251942e3be78c8c599c46488)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.125913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.126427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.126603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.126883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.127153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.128286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:38.128286 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.129679 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.143128 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1227/2460] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:32:38.148069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.158864 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b44d6e-a8f7-4dd4-a9a8-01dc874bb4d3 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.166005 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.166616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.166890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.167188 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.167820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.169909 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.181863 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6957bf53251942e3be78c8c599c46488', 'request_id': 'req-88f0965f-c3b8-4b7a-8f35-64342080124e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.189713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6957bf53251942e3be78c8c599c46488', 'request_id': 'req-88f0965f-c3b8-4b7a-8f35-64342080124e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.196250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6957bf53251942e3be78c8c599c46488', 'request_id': 'req-88f0965f-c3b8-4b7a-8f35-64342080124e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.206717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1228/2461] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:38.208194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.209055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] The token cache is being invalidate because project 6957bf53251942e3be78c8c599c46488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:38.209347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.210256 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.211914 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1234/2462] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:38 2022] DELETE /identity/v3/projects/6957bf53251942e3be78c8c599c46488 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:38.215683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.217224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.220824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:38.281871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98cdbd-816a-4e97-8af8-453735404b4b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.286511 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.287723 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.288004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.288298 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.288573 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0965f-c3b8-4b7a-8f35-64342080124e admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.288919 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.290963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.293894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.294577 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.294853 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.295123 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] PATH_INFO: `/v3/users/f6edb40cc3544e29b67dfc35415e100a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.296362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6edb40cc3544e29b67dfc35415e100a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.298266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.315169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1229/2463] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:38.319244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.321008 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.335905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e1381-07ee-40ee-aa82-9c8163960464 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.340682 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.341276 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.341551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.341818 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.342449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.344028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.348495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6edb40cc3544e29b67dfc35415e100a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.355647 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1230/2464] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:38.359588 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.361317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6edb40cc3544e29b67dfc35415e100a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.362011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6edb40cc3544e29b67dfc35415e100a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.368756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6edb40cc3544e29b67dfc35415e100a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.369386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884efc1-a794-4821-8642-7922170e8fc9 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.374742 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1926-14c1-41a7-a197-58d7fc9f5df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.375410 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1926-14c1-41a7-a197-58d7fc9f5df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.375664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1926-14c1-41a7-a197-58d7fc9f5df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.375936 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1926-14c1-41a7-a197-58d7fc9f5df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.376583 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1926-14c1-41a7-a197-58d7fc9f5df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.377116 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6edb40cc3544e29b67dfc35415e100a'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.378261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1926-14c1-41a7-a197-58d7fc9f5df3 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.384473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1235/2465] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:38 2022] DELETE /identity/v3/users/f6edb40cc3544e29b67dfc35415e100a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:38.393284 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1231/2466] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:38.400150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.401895 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.407458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:38.488673 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226cacf-dfce-4246-b14c-6b26ee2934b6 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.493680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.494369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.494634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.494899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] PATH_INFO: `/v3/projects/0a9133c3823f403dbb07ff14599e5b15/users/604bfa617dba495cbb8fad263ed6fc9a/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.511315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9133c3823f403dbb07ff14599e5b15, user_id=604bfa617dba495cbb8fad263ed6fc9a, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.513962 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.530412 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1236/2467] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:38 2022] PUT /identity/v3/projects/0a9133c3823f403dbb07ff14599e5b15/users/604bfa617dba495cbb8fad263ed6fc9a/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:38.547774 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.548058 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.548394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.577452 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] MFA Rules not processed for user `604bfa617dba495cbb8fad263ed6fc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:38.578628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.655052 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1232/2468] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:38.659713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.707743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.710236 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.751390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfffc2f-a4f3-4a2e-abba-78cedcdabe0b admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.756710 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.757403 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.757736 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.758064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.758780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.760728 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.783169 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bbd51-1222-44a8-a14c-18f2a8e366c2 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.788130 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.788318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1237/2469] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:38.788798 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.789140 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.789512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] PATH_INFO: `/v3/projects/ff3e0f4e122b41e3bcc5cd637c970168` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.790730 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3e0f4e122b41e3bcc5cd637c970168)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.792342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.792725 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.799532 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:38.843726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3e0f4e122b41e3bcc5cd637c970168', 'request_id': 'req-75861b13-6397-4b41-b775-14cc6daa3dae'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.852717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3e0f4e122b41e3bcc5cd637c970168', 'request_id': 'req-75861b13-6397-4b41-b775-14cc6daa3dae'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.858463 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3e0f4e122b41e3bcc5cd637c970168', 'request_id': 'req-75861b13-6397-4b41-b775-14cc6daa3dae'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.859686 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e954-58f0-4b30-98b7-4e10bacec1f3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.870655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] The token cache is being invalidate because project ff3e0f4e122b41e3bcc5cd637c970168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:38.870947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.871622 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-75861b13-6397-4b41-b775-14cc6daa3dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:38.873064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:38.873238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1233/2470] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:38 2022] DELETE /identity/v3/projects/ff3e0f4e122b41e3bcc5cd637c970168 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:38.878287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24d5b-55d9-4a57-8b51-a79caa2243c5 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.878596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24d5b-55d9-4a57-8b51-a79caa2243c5 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.878868 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24d5b-55d9-4a57-8b51-a79caa2243c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.900965 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-0ea24d5b-55d9-4a57-8b51-a79caa2243c5 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:38.902230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0ea24d5b-55d9-4a57-8b51-a79caa2243c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.928898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.929759 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.930316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.930723 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.931542 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.934893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.960588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1238/2471] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/users => generated 301 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:38.964611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:38.966204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:38.966867 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1234/2472] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:38.976034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddaa8d-6f31-4ae0-8e7c-84c59fe331e0 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.976424 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddaa8d-6f31-4ae0-8e7c-84c59fe331e0 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.976619 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddaa8d-6f31-4ae0-8e7c-84c59fe331e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.979613 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55df4-fed2-4f8c-97bf-3205c66a6203 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:38.984315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:38.984921 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:38.985358 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:38.985820 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:38.986407 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:38.988018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:38.997900 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-02ddaa8d-6f31-4ae0-8e7c-84c59fe331e0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:38.999055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-02ddaa8d-6f31-4ae0-8e7c-84c59fe331e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:39.000298 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1239/2473] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:39.004482 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.008913 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1235/2474] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:39.014666 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddaa8d-6f31-4ae0-8e7c-84c59fe331e0 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.015448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3039ecb-3ad8-4ff9-a0ee-0539ae93edc1 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.020167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.020874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.021150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.021474 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] PATH_INFO: `/v3/projects/012bba67f2784ce68310e809890b59c2/users/34a75c2e06ed450389bc36bf2150282c/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.024659 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddaa8d-6f31-4ae0-8e7c-84c59fe331e0 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.029314 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.029923 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.030202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.030472 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.031116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:39.031442 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.033287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.037359 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=012bba67f2784ce68310e809890b59c2, user_id=34a75c2e06ed450389bc36bf2150282c, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.040209 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.046577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1236/2475] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 27 13:32:39.051092 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.054920 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1240/2476] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:39 2022] PUT /identity/v3/projects/012bba67f2784ce68310e809890b59c2/users/34a75c2e06ed450389bc36bf2150282c/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:39.058643 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.062299 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f3e6d-24b4-4bca-998f-baa05a5d3d0e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.124978 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8629f8-d621-488a-9fed-46b4c512b3f2 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.129854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.130090 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.131452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.131452 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.131772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.131964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.132185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.132498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.132817 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.133646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.133799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.135776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.144150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1241/2477] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:39.148589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.158871 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e91e0-7672-4406-bf3b-de6f737a4f45 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.160842 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:39.164827 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27976e0c-e016-4071-bce1-ae48d6dae46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.165491 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27976e0c-e016-4071-bce1-ae48d6dae46c admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.165772 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27976e0c-e016-4071-bce1-ae48d6dae46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.166047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27976e0c-e016-4071-bce1-ae48d6dae46c admin admin] PATH_INFO: `/v3/projects/012bba67f2784ce68310e809890b59c2/users/34a75c2e06ed450389bc36bf2150282c/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.168040 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27976e0c-e016-4071-bce1-ae48d6dae46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=012bba67f2784ce68310e809890b59c2, user_id=34a75c2e06ed450389bc36bf2150282c, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.169082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1237/2478] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:39 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:39.170816 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27976e0c-e016-4071-bce1-ae48d6dae46c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.173111 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.182614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1242/2479] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:39 2022] PUT /identity/v3/projects/012bba67f2784ce68310e809890b59c2/users/34a75c2e06ed450389bc36bf2150282c/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:39.195580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.196085 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.196510 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.227801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] MFA Rules not processed for user `34a75c2e06ed450389bc36bf2150282c`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:39.229056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:39.238259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54bf96-df1d-4963-a9fa-6ddb4a937d9b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.279466 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:39.296514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.297129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.297404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.297677 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.298359 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.300288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.305437 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1243/2480] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:39 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:39.316525 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.318035 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:39.323560 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1238/2481] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:39 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:39.327646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.337777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c41f95-48e6-43f9-b022-1d33693d169b admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.342646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.343243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.343517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.343786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.344461 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.346065 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.356655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1239/2482] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:39.360641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.370952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b84e2b-2396-46ff-aae6-6cff2ab75cb8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.375734 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.376423 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.376775 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.377043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.377673 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.378980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaefb7d-00a4-42df-827c-caba9ec45b62 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.379439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.382821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb55bb-b302-495d-a689-a0c42f7eaa9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.383435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb55bb-b302-495d-a689-a0c42f7eaa9b service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.383720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb55bb-b302-495d-a689-a0c42f7eaa9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.383987 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb55bb-b302-495d-a689-a0c42f7eaa9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.394142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1240/2483] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:39.394373 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb55bb-b302-495d-a689-a0c42f7eaa9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.395257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.395661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb55bb-b302-495d-a689-a0c42f7eaa9b service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.405015 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbc12a-1cee-49f7-992a-043b374f3806 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.405666 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1244/2484] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:39.409719 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.410466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.410762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.411031 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] PATH_INFO: `/v3/projects/440e66b237bd4d1ea46d061f05e9f830/users/2ee9ff432217444b995b1a6f042629fc/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.432473 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] RBAC: Authorizing `identity:create_grant(project_id=440e66b237bd4d1ea46d061f05e9f830, user_id=2ee9ff432217444b995b1a6f042629fc, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.435162 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.450856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1241/2485] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:39 2022] PUT /identity/v3/projects/440e66b237bd4d1ea46d061f05e9f830/users/2ee9ff432217444b995b1a6f042629fc/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:39.461322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d0f5-0301-43d8-974e-6e04d6ea0384 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.461701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d0f5-0301-43d8-974e-6e04d6ea0384 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.462162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d0f5-0301-43d8-974e-6e04d6ea0384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.492207 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3f93d0f5-0301-43d8-974e-6e04d6ea0384 None None] MFA Rules not processed for user `2ee9ff432217444b995b1a6f042629fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:39.493370 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3f93d0f5-0301-43d8-974e-6e04d6ea0384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:39.555338 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.566685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1245/2486] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:39.571215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93d0f5-0301-43d8-974e-6e04d6ea0384 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.617505 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:39.630507 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39e888-24f5-4406-8f5c-11c714ca453f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.637812 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.638459 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.638736 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.639082 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] PATH_INFO: `/v3/users/604bfa617dba495cbb8fad263ed6fc9a` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.640139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=604bfa617dba495cbb8fad263ed6fc9a)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.640443 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93d0f5-0301-43d8-974e-6e04d6ea0384 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.642158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.650086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.650735 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.651055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.651325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.652172 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.654193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.676206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:39.685593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1246/2487] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:39 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:39.689390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.693492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604bfa617dba495cbb8fad263ed6fc9a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:39.706462 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604bfa617dba495cbb8fad263ed6fc9a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:39.707322 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604bfa617dba495cbb8fad263ed6fc9a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:39.714469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604bfa617dba495cbb8fad263ed6fc9a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:39.723278 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-9ee9fdcc-3568-44bc-ba33-fb3043933193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604bfa617dba495cbb8fad263ed6fc9a'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:39.730051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1242/2488] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:39 2022] DELETE /identity/v3/users/604bfa617dba495cbb8fad263ed6fc9a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:39.734366 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.734649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.734914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.753488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322078a1-2f0b-46de-82ea-3737d9d7c46c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.759966 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:39.762263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:39.814488 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.815106 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.815387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.815653 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.816357 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.818315 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.824335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1243/2489] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:39.835496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.836916 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:39.843806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1247/2490] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:39 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:39.844967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.846576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:39.860517 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06337431-a31c-46cd-95cf-9bd2a5cac219 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.865973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.866609 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.866887 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.867193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.867936 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.869563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.880578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1248/2491] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:39.884722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.894960 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f886d-46f3-4621-9fd7-6d1012ccc8f4 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.897827 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba6745-36af-4c1a-a391-06a84a1e9677 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.900156 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.900831 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.901107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.901369 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] PATH_INFO: `/v3/projects/23a10428400742e78d0c7a2ea37af2cd/users/31ae9376963a4cb3a35aafc4e1932c2f/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.901562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:39.902182 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:39.902457 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:39.902728 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:39.914961 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.916435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.918659 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a10428400742e78d0c7a2ea37af2cd, user_id=31ae9376963a4cb3a35aafc4e1932c2f, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:39.921320 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:39.926477 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1244/2492] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:39.935143 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1249/2493] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:39 2022] PUT /identity/v3/projects/23a10428400742e78d0c7a2ea37af2cd/users/31ae9376963a4cb3a35aafc4e1932c2f/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:39.939063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:39.948885 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcfff3-8e1b-4710-a2ba-ff2a8c5e98de service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:39.958337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:40.013783 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942ab4c-9fce-445d-9711-9c1702925cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:40.014313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ab4c-9fce-445d-9711-9c1702925cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:40.014595 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ab4c-9fce-445d-9711-9c1702925cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:40.014898 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ab4c-9fce-445d-9711-9c1702925cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:40.015540 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942ab4c-9fce-445d-9711-9c1702925cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:40.017173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942ab4c-9fce-445d-9711-9c1702925cb9 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:40.028369 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1245/2494] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:40.032671 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:40.043905 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b90fb-6f05-4cb0-9377-b77e1e29a371 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:40.049257 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:40.049917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:40.050197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:40.050467 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] PATH_INFO: `/v3/projects/23a10428400742e78d0c7a2ea37af2cd/users/31ae9376963a4cb3a35aafc4e1932c2f/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:40.052528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a10428400742e78d0c7a2ea37af2cd, user_id=31ae9376963a4cb3a35aafc4e1932c2f, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:40.055145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:40.066600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1250/2495] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:40 2022] PUT /identity/v3/projects/23a10428400742e78d0c7a2ea37af2cd/users/31ae9376963a4cb3a35aafc4e1932c2f/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:40.077180 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:40.077467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:40.077740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:40.112313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] MFA Rules not processed for user `31ae9376963a4cb3a35aafc4e1932c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:40.113735 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:40.185809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1246/2496] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:40.197626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:40.199361 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:40.217794 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:40.219303 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:40.260704 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:40.266167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2661502-cc38-4e17-a9ef-b7ecf56bd892 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:40.270958 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:40.272122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:40.272445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:40.272717 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:40.274528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:40.292100 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d98442-9c66-4502-8bec-44e0656c168b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:40.296853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:40.297485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:40.297764 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:40.298036 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] PATH_INFO: `/v3/projects/0a9133c3823f403dbb07ff14599e5b15` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:40.299178 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9133c3823f403dbb07ff14599e5b15)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:40.301179 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:40.337199 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:40.338670 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:40.342095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:40.348413 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1251/2497] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:40.354483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9133c3823f403dbb07ff14599e5b15', 'request_id': 'req-25be38a1-bcbc-48d6-93fe-18069cfc2856'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.362200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9133c3823f403dbb07ff14599e5b15', 'request_id': 'req-25be38a1-bcbc-48d6-93fe-18069cfc2856'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.368492 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9133c3823f403dbb07ff14599e5b15', 'request_id': 'req-25be38a1-bcbc-48d6-93fe-18069cfc2856'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.382565 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] The token cache is being invalidate because project 0a9133c3823f403dbb07ff14599e5b15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:40.382891 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.384220 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.386121 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1247/2498] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:40 2022] DELETE /identity/v3/projects/0a9133c3823f403dbb07ff14599e5b15 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:40.391600 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:40.404441 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db41b8-1a17-4f27-90d3-6023ad365600 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:40.471646 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:40.472505 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:40.472908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:40.473286 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] PATH_INFO: `/v3/users/34a75c2e06ed450389bc36bf2150282c` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:40.474581 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a75c2e06ed450389bc36bf2150282c)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:40.476565 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:40.525528 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a75c2e06ed450389bc36bf2150282c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.537961 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a75c2e06ed450389bc36bf2150282c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.538766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a75c2e06ed450389bc36bf2150282c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.546130 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a75c2e06ed450389bc36bf2150282c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.554442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-01cd0b0f-a8ad-451d-825e-e3aeac6e11d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a75c2e06ed450389bc36bf2150282c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:40.560913 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1252/2499] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:40 2022] DELETE /identity/v3/users/34a75c2e06ed450389bc36bf2150282c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:40.894414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:40.895831 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:40.912420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be38a1-bcbc-48d6-93fe-18069cfc2856 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:40.974790 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:40.975432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:40.975709 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:40.975986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] PATH_INFO: `/v3/projects/012bba67f2784ce68310e809890b59c2` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:40.977232 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012bba67f2784ce68310e809890b59c2)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:40.979167 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.030811 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012bba67f2784ce68310e809890b59c2', 'request_id': 'req-8c92e6a8-917c-4c73-871d-f64ab990d911'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:41.038292 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012bba67f2784ce68310e809890b59c2', 'request_id': 'req-8c92e6a8-917c-4c73-871d-f64ab990d911'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:41.044299 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012bba67f2784ce68310e809890b59c2', 'request_id': 'req-8c92e6a8-917c-4c73-871d-f64ab990d911'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:41.056674 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] The token cache is being invalidate because project 012bba67f2784ce68310e809890b59c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:41.056947 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:41.057582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-8c92e6a8-917c-4c73-871d-f64ab990d911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:41.059116 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1248/2500] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:40 2022] DELETE /identity/v3/projects/012bba67f2784ce68310e809890b59c2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:41.064901 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8752d8-cf7d-461e-aa8b-3fb99d2d0361 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.065191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8752d8-cf7d-461e-aa8b-3fb99d2d0361 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.065464 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8752d8-cf7d-461e-aa8b-3fb99d2d0361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.085637 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-ee8752d8-cf7d-461e-aa8b-3fb99d2d0361 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:41.086912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ee8752d8-cf7d-461e-aa8b-3fb99d2d0361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:41.100619 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:41.150594 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1253/2501] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:41.160004 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836898bb-c0a4-493c-abc0-b721d6d476ad None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.160326 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836898bb-c0a4-493c-abc0-b721d6d476ad None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.160603 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836898bb-c0a4-493c-abc0-b721d6d476ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.183593 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-836898bb-c0a4-493c-abc0-b721d6d476ad None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:41.185119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-836898bb-c0a4-493c-abc0-b721d6d476ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:41.193986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1249/2502] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:41.198478 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8752d8-cf7d-461e-aa8b-3fb99d2d0361 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.208984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8752d8-cf7d-461e-aa8b-3fb99d2d0361 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.214311 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.214933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.215206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.215470 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.216109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:41.216446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.218047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.230630 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1254/2503] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:41.235414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836898bb-c0a4-493c-abc0-b721d6d476ad None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.246080 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836898bb-c0a4-493c-abc0-b721d6d476ad None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.250941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.251543 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.251808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.252072 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.252738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.254639 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.282246 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1250/2504] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:41 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:41.286461 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.297288 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c833ec7-be42-46b0-bdc9-5cd6e55a7bcd admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.355397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.356004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.356301 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.356567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.357192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.359165 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.380897 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1255/2505] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:41 2022] POST /identity/v3/users => generated 305 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:41.386642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.398914 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27785a-f601-436a-9086-bb5958ba4302 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.404064 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.404738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.405017 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.405288 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.406028 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.407614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.418792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1251/2506] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:41.423614 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.434418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8b0bb-c1a1-4d32-829f-b7c9024b9b03 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.441506 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a824-c8ae-4019-aca7-eeb35ab71765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.442518 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a824-c8ae-4019-aca7-eeb35ab71765 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.443183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a824-c8ae-4019-aca7-eeb35ab71765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.443513 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a824-c8ae-4019-aca7-eeb35ab71765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.444733 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a824-c8ae-4019-aca7-eeb35ab71765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.447782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a824-c8ae-4019-aca7-eeb35ab71765 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.466804 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1256/2507] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:41.471057 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.482011 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1985-3e3c-46d3-b6f9-265ada42f916 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.486630 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.487266 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.487625 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.487897 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] PATH_INFO: `/v3/projects/c246c855f0384ea19109b0c2f2ba61db/users/4784c290118749e496c4e8b33ef4a9ba/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.501041 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:41.504644 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c246c855f0384ea19109b0c2f2ba61db, user_id=4784c290118749e496c4e8b33ef4a9ba, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.508297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.526822 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1252/2508] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:41 2022] PUT /identity/v3/projects/c246c855f0384ea19109b0c2f2ba61db/users/4784c290118749e496c4e8b33ef4a9ba/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:41.539417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb391f-955c-4dd0-9398-6ffd240daaae None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.539627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb391f-955c-4dd0-9398-6ffd240daaae None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.539894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb391f-955c-4dd0-9398-6ffd240daaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.568087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9dbb391f-955c-4dd0-9398-6ffd240daaae None None] MFA Rules not processed for user `4784c290118749e496c4e8b33ef4a9ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:41.569318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9dbb391f-955c-4dd0-9398-6ffd240daaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:41.643760 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1257/2509] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:41 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:41.649038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.712873 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6936-3c70-4d2a-b5c4-79472b1d56f0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.717660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.718259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.718534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.718803 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.719414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.721335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.747984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1253/2510] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:41 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:41.752507 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb391f-955c-4dd0-9398-6ffd240daaae None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.763706 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb391f-955c-4dd0-9398-6ffd240daaae None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.776181 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:41.825325 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.826334 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.826799 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.827269 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] PATH_INFO: `/v3/users` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.828611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.831943 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.867975 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1258/2511] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:41 2022] POST /identity/v3/users => generated 311 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:41.869271 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.880534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c1bde-bbe4-4c3e-ad7d-6f6b094ea7d0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.885693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.886297 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.888002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.888323 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.888977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.890589 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.901960 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1254/2512] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:41.906591 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.918050 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90f451-878d-44dc-86c9-475d982bbe56 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:41.923196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:41.923849 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:41.924265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:41.924580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] PATH_INFO: `/v3/projects/4fef0e07f9ea4afeb31f25991dc62818/users/764299af787044278678a4c3b924e636/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:41.942504 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fef0e07f9ea4afeb31f25991dc62818, user_id=764299af787044278678a4c3b924e636, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:41.946064 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:41.960347 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1259/2513] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:41 2022] PUT /identity/v3/projects/4fef0e07f9ea4afeb31f25991dc62818/users/764299af787044278678a4c3b924e636/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:41.964693 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:41.976084 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5e93d-759a-4951-9a82-f685d5b40d44 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:42.040990 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195175f-4cc6-47fa-bd3d-563f2189db46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:42.041598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195175f-4cc6-47fa-bd3d-563f2189db46 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:42.041876 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195175f-4cc6-47fa-bd3d-563f2189db46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:42.042155 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195175f-4cc6-47fa-bd3d-563f2189db46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:42.042802 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195175f-4cc6-47fa-bd3d-563f2189db46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:42.044618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195175f-4cc6-47fa-bd3d-563f2189db46 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:42.055398 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1255/2514] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 27 13:32:42.061372 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:42.075665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c379-4ac3-4dd9-af67-591bba473e98 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:42.084056 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:42.084794 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:42.085073 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:42.085340 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] PATH_INFO: `/v3/projects/4fef0e07f9ea4afeb31f25991dc62818/users/764299af787044278678a4c3b924e636/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:42.087342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fef0e07f9ea4afeb31f25991dc62818, user_id=764299af787044278678a4c3b924e636, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:42.090080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:42.102304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1260/2515] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:42 2022] PUT /identity/v3/projects/4fef0e07f9ea4afeb31f25991dc62818/users/764299af787044278678a4c3b924e636/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:42.113243 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6a696-278a-4909-9f2a-00842b3e9708 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:42.113536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6a696-278a-4909-9f2a-00842b3e9708 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:42.113789 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6a696-278a-4909-9f2a-00842b3e9708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:42.143983 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-86d6a696-278a-4909-9f2a-00842b3e9708 None None] MFA Rules not processed for user `764299af787044278678a4c3b924e636`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:42.145396 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-86d6a696-278a-4909-9f2a-00842b3e9708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:42.207221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:42.233534 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1256/2516] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:42 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:42.247272 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:42.249561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:42.318899 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf93f3-7474-49a2-b231-069391602e43 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:42.322694 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:42.323363 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:42.323638 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:42.323908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:42.334610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:42.336155 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:42.346541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1261/2517] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:42 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:42.520475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6a696-278a-4909-9f2a-00842b3e9708 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:42.531231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6a696-278a-4909-9f2a-00842b3e9708 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:42.535183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:42.535785 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:42.536149 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:42.536447 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:42.603021 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:42.604470 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:42.613769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1257/2518] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:42 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:44.792511 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:44.793915 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:44.805020 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916e740-865e-4d87-8121-1f202f0a0fd8 service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:44.808610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c90eeb-68dd-4ef3-a4c5-d54c26de7059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:44.809196 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c90eeb-68dd-4ef3-a4c5-d54c26de7059 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:44.809475 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c90eeb-68dd-4ef3-a4c5-d54c26de7059 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:44.809746 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c90eeb-68dd-4ef3-a4c5-d54c26de7059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:44.811390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-83c90eeb-68dd-4ef3-a4c5-d54c26de7059 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:44.874327 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c90eeb-68dd-4ef3-a4c5-d54c26de7059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:44.875747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c90eeb-68dd-4ef3-a4c5-d54c26de7059 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:44.885192 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1262/2519] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:50.016411 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:50.078809 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafa325-d492-4c2c-8471-d437853ccf07 service cinder] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:50.083905 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:50.084564 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:50.084837 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:50.085132 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] PATH_INFO: `/v3/users/4784c290118749e496c4e8b33ef4a9ba` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:50.086238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4784c290118749e496c4e8b33ef4a9ba)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:50.088135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:50.133861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4784c290118749e496c4e8b33ef4a9ba'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.146436 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4784c290118749e496c4e8b33ef4a9ba'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.147033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4784c290118749e496c4e8b33ef4a9ba'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.153838 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4784c290118749e496c4e8b33ef4a9ba'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.161598 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4784c290118749e496c4e8b33ef4a9ba'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.167550 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1258/2520] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:50 2022] DELETE /identity/v3/users/4784c290118749e496c4e8b33ef4a9ba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:50.172341 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:50.172888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:50.172996 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:50.181160 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:50.200205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:50.201431 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:50.259182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1263/2521] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:50.268823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:50.270339 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:50.276865 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:50.331230 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87992c11-6aa3-41eb-8470-7953d7bd0ac8 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:50.334954 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:50.335552 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:50.335823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:50.336089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:50.345025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:50.346432 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:50.355660 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1259/2522] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:50.621520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:50.623045 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:50.636660 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dea87-e20b-4651-a9f6-fbd6df779007 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:50.641549 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:50.642193 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:50.642471 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:50.642749 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] PATH_INFO: `/v3/projects/c246c855f0384ea19109b0c2f2ba61db` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:50.643930 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c246c855f0384ea19109b0c2f2ba61db)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:50.645908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:50.693396 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c246c855f0384ea19109b0c2f2ba61db', 'request_id': 'req-32263ec8-5483-4061-8e74-fafc6f344112'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.701247 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c246c855f0384ea19109b0c2f2ba61db', 'request_id': 'req-32263ec8-5483-4061-8e74-fafc6f344112'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.707497 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c246c855f0384ea19109b0c2f2ba61db', 'request_id': 'req-32263ec8-5483-4061-8e74-fafc6f344112'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.718641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] The token cache is being invalidate because project c246c855f0384ea19109b0c2f2ba61db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:50.718928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.719620 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.721479 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1264/2523] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:50 2022] DELETE /identity/v3/projects/c246c855f0384ea19109b0c2f2ba61db => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:50.725952 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:50.736434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963d0ad-2f5a-4d5b-bdcf-23a3a4647005 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:50.793158 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:50.793784 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:50.794061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:50.794330 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] PATH_INFO: `/v3/users/764299af787044278678a4c3b924e636` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:50.795428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=764299af787044278678a4c3b924e636)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:50.797370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:50.828766 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:50.830349 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:50.843756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32263ec8-5483-4061-8e74-fafc6f344112 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:50.844397 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764299af787044278678a4c3b924e636'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.849081 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:50.849779 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:50.850127 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:50.850473 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] PATH_INFO: `/v3/users/2ee9ff432217444b995b1a6f042629fc` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:50.851663 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee9ff432217444b995b1a6f042629fc)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:50.853661 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:50.857194 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764299af787044278678a4c3b924e636'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.857866 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764299af787044278678a4c3b924e636'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.865614 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764299af787044278678a4c3b924e636'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.874018 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-e70d0a2c-e71c-4bd5-9c5c-6ce34693d38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764299af787044278678a4c3b924e636'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.880085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1260/2524] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:50 2022] DELETE /identity/v3/users/764299af787044278678a4c3b924e636 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:50.907587 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee9ff432217444b995b1a6f042629fc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.921809 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee9ff432217444b995b1a6f042629fc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.922592 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee9ff432217444b995b1a6f042629fc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.930182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee9ff432217444b995b1a6f042629fc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.937593 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee9ff432217444b995b1a6f042629fc'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:50.940150 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:50.945253 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1265/2525] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:50 2022] DELETE /identity/v3/users/2ee9ff432217444b995b1a6f042629fc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:50.949879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:50.950173 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:50.950449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:50.971551 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:50.972703 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:50.995370 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:51.039238 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1261/2526] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:51.048855 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.050624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.112521 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda0bc9-8a30-4e1d-8ccf-8ad96ad4233d admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.116183 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2927fdc-e2c9-4ee7-ad2b-36b9c533143f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.116850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927fdc-e2c9-4ee7-ad2b-36b9c533143f service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.117124 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927fdc-e2c9-4ee7-ad2b-36b9c533143f service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.117387 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927fdc-e2c9-4ee7-ad2b-36b9c533143f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.127386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2927fdc-e2c9-4ee7-ad2b-36b9c533143f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.128896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2927fdc-e2c9-4ee7-ad2b-36b9c533143f service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.138503 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1266/2527] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:51.174073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.175950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.195183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e9779-722a-42f7-ab7c-19dc7dbb9afc None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.201668 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.202471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.202791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.203050 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] PATH_INFO: `/v3/projects/4fef0e07f9ea4afeb31f25991dc62818` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.204293 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fef0e07f9ea4afeb31f25991dc62818)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.206331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.255395 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fef0e07f9ea4afeb31f25991dc62818', 'request_id': 'req-57ecf605-d98d-4dae-9dfb-3472e828c84f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.262791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fef0e07f9ea4afeb31f25991dc62818', 'request_id': 'req-57ecf605-d98d-4dae-9dfb-3472e828c84f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.269675 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fef0e07f9ea4afeb31f25991dc62818', 'request_id': 'req-57ecf605-d98d-4dae-9dfb-3472e828c84f'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.282649 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] The token cache is being invalidate because project 4fef0e07f9ea4afeb31f25991dc62818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:51.282950 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.283853 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-57ecf605-d98d-4dae-9dfb-3472e828c84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.285382 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1262/2528] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:51 2022] DELETE /identity/v3/projects/4fef0e07f9ea4afeb31f25991dc62818 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:51.291556 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.292115 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.293018 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.314997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:51.316200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.377783 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1267/2529] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:51.386184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9bced-a90d-4996-843c-f69a71ff7b88 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.386539 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9bced-a90d-4996-843c-f69a71ff7b88 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.387767 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9bced-a90d-4996-843c-f69a71ff7b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.408034 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-55c9bced-a90d-4996-843c-f69a71ff7b88 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:51.409273 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-55c9bced-a90d-4996-843c-f69a71ff7b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.419856 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.420556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1263/2530] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:51.421417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.422506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9bced-a90d-4996-843c-f69a71ff7b88 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.433267 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9bced-a90d-4996-843c-f69a71ff7b88 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.433406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95214202-5604-4e9f-b2ce-fca09f59057e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.438215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.438420 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.438875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.439044 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.439158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.439325 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.439427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] PATH_INFO: `/v3/projects/440e66b237bd4d1ea46d061f05e9f830` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.439641 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.440331 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:51.440519 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=440e66b237bd4d1ea46d061f05e9f830)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.440656 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.442823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.442952 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.460287 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1264/2531] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:51.461434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.471435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc31ea3-3799-4ff5-9c78-1eb65686409d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.476846 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.477471 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.477749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.478037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] PATH_INFO: `/v3/projects` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.478653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.480586 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.491118 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440e66b237bd4d1ea46d061f05e9f830', 'request_id': 'req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.499046 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440e66b237bd4d1ea46d061f05e9f830', 'request_id': 'req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.506292 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440e66b237bd4d1ea46d061f05e9f830', 'request_id': 'req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.510620 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1265/2532] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:51 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:51.514968 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.518057 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] The token cache is being invalidate because project 440e66b237bd4d1ea46d061f05e9f830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:51.518390 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.519213 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.521567 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1268/2533] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:51 2022] DELETE /identity/v3/projects/440e66b237bd4d1ea46d061f05e9f830 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:51.525158 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.526750 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.557271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:51.580068 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc2c04-7976-4acb-80ae-ccd032ea7981 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.585780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.586386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.586658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.586935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.587537 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.589509 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.596318 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4d028-cd3d-4f7d-9df6-685bc4fbaff3 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.601187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.601821 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.602107 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.602401 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] PATH_INFO: `/v3/users/31ae9376963a4cb3a35aafc4e1932c2f` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.603695 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ae9376963a4cb3a35aafc4e1932c2f)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.605811 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.606135 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:51.615753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1266/2534] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:51 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:51.617016 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.618877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.631926 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d3d7-7fd9-4550-9627-e7db37a9ae63 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.637259 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.637907 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.638184 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.638446 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.639079 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.640929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.654895 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1267/2535] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:51.656105 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.664080 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ae9376963a4cb3a35aafc4e1932c2f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.666445 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea480-9e18-49e4-abff-5c10fb0e63bd admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.672037 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.672663 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.672934 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.673202 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.673857 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.675506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.676924 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ae9376963a4cb3a35aafc4e1932c2f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.677664 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ae9376963a4cb3a35aafc4e1932c2f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.686206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ae9376963a4cb3a35aafc4e1932c2f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.691177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1268/2536] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:51.693122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.695060 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-bb43a9a9-95d4-41b8-8e52-5ef5f1b8477d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ae9376963a4cb3a35aafc4e1932c2f'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:51.701950 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1269/2537] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:51 2022] DELETE /identity/v3/users/31ae9376963a4cb3a35aafc4e1932c2f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:51.703313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488856d-11d7-4942-92d0-f7ff50cbdf91 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:51.763379 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:51.763953 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.764254 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.764524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] PATH_INFO: `/v3/projects/a99fae3350634a698293df70ed2b881a/users/00f25028b1f44112a8f8ed93b24c127d/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.780006 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99fae3350634a698293df70ed2b881a, user_id=00f25028b1f44112a8f8ed93b24c127d, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:51.782888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:51.797231 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1269/2538] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:51 2022] PUT /identity/v3/projects/a99fae3350634a698293df70ed2b881a/users/00f25028b1f44112a8f8ed93b24c127d/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:51.865423 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:51.865634 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:51.865911 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:51.895110 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] MFA Rules not processed for user `00f25028b1f44112a8f8ed93b24c127d`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:51.897086 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.963484 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1270/2539] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:51.975530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.977524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:51.987912 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:51.990093 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:52.036550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:52.046980 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf596-0398-4a27-944e-b0a24233501e admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.050888 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923b6db-346c-4ff4-bd0f-6cc6001c2e8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.052043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b6db-346c-4ff4-bd0f-6cc6001c2e8b service cinder] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.052389 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b6db-346c-4ff4-bd0f-6cc6001c2e8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.052618 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b6db-346c-4ff4-bd0f-6cc6001c2e8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.061353 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923b6db-346c-4ff4-bd0f-6cc6001c2e8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.062779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923b6db-346c-4ff4-bd0f-6cc6001c2e8b service cinder] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.065917 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b9977-cc9d-469c-860e-aecf5fd0b068 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.071197 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.071825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.072101 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.072581 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1270/2540] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:32:52.072722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] PATH_INFO: `/v3/projects/23a10428400742e78d0c7a2ea37af2cd` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.073520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a10428400742e78d0c7a2ea37af2cd)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.075425 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.124458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a10428400742e78d0c7a2ea37af2cd', 'request_id': 'req-66480593-697d-4c2b-a995-9327fa5b795e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:52.136935 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a10428400742e78d0c7a2ea37af2cd', 'request_id': 'req-66480593-697d-4c2b-a995-9327fa5b795e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:52.143415 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a10428400742e78d0c7a2ea37af2cd', 'request_id': 'req-66480593-697d-4c2b-a995-9327fa5b795e'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:52.155132 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] The token cache is being invalidate because project 23a10428400742e78d0c7a2ea37af2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:52.155442 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:52.156188 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-66480593-697d-4c2b-a995-9327fa5b795e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:52.157756 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1271/2541] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:51 2022] DELETE /identity/v3/projects/23a10428400742e78d0c7a2ea37af2cd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:52.164206 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910cf6d-1c93-49f1-92f1-2c4cb312e137 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.164512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910cf6d-1c93-49f1-92f1-2c4cb312e137 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.164779 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910cf6d-1c93-49f1-92f1-2c4cb312e137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.185421 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-1910cf6d-1c93-49f1-92f1-2c4cb312e137 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:52.186820 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1910cf6d-1c93-49f1-92f1-2c4cb312e137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:52.248878 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1271/2542] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 27 13:32:52.257672 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1edca-7293-4c76-8cc5-aaaf14115c9e None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.257949 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1edca-7293-4c76-8cc5-aaaf14115c9e None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.258228 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1edca-7293-4c76-8cc5-aaaf14115c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.279701 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-27e1edca-7293-4c76-8cc5-aaaf14115c9e None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:52.280858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-27e1edca-7293-4c76-8cc5-aaaf14115c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:52.290568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1272/2543] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:52.294596 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910cf6d-1c93-49f1-92f1-2c4cb312e137 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.299664 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:52.306672 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910cf6d-1c93-49f1-92f1-2c4cb312e137 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.311439 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.312042 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.312365 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.312661 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.313263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:32:52.313562 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.315122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.327381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1272/2544] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:32:52.332036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1edca-7293-4c76-8cc5-aaaf14115c9e None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.341540 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1edca-7293-4c76-8cc5-aaaf14115c9e None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.346780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.347417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.347693 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.347969 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.349185 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.351212 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.379427 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1273/2545] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:32:52 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:32:52.385655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.418720 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef420628-c21f-4168-a78f-7266de5b7e77 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.480051 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.480908 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.481261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.481645 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.482383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.484599 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.508805 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1273/2546] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:32:52 2022] POST /identity/v3/users => generated 279 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:32:52.512845 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.523885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856d5b-e368-4be1-9aae-0da2a3672ba8 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.528458 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.529047 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.529323 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.529589 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.530266 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.531780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.542455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1274/2547] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:52.546524 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.556600 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253579bb-6d99-49dd-b3a6-2e53441d5431 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.561163 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f532b5e-92d3-4e8a-b335-6f38601217ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.561754 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f532b5e-92d3-4e8a-b335-6f38601217ae admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.562035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f532b5e-92d3-4e8a-b335-6f38601217ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.562307 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f532b5e-92d3-4e8a-b335-6f38601217ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.562941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f532b5e-92d3-4e8a-b335-6f38601217ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.564590 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f532b5e-92d3-4e8a-b335-6f38601217ae admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.575249 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1274/2548] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:32:52.579191 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.589493 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a801c2-0e32-4255-863e-2a141bd1e357 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.594215 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.594868 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.595146 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.595420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] PATH_INFO: `/v3/projects/17efd28d21a34bed81fa4f8b4eaac04e/users/fb7ac44555d744d49c27bfb70cb368d5/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.610611 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17efd28d21a34bed81fa4f8b4eaac04e, user_id=fb7ac44555d744d49c27bfb70cb368d5, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.613296 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.626815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1275/2549] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:32:52 2022] PUT /identity/v3/projects/17efd28d21a34bed81fa4f8b4eaac04e/users/fb7ac44555d744d49c27bfb70cb368d5/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 27 13:32:52.638198 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ee8c7-8e3d-4e45-bfb9-8f203a61cef3 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.638479 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ee8c7-8e3d-4e45-bfb9-8f203a61cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.638750 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ee8c7-8e3d-4e45-bfb9-8f203a61cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.667251 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-4c5ee8c7-8e3d-4e45-bfb9-8f203a61cef3 None None] MFA Rules not processed for user `fb7ac44555d744d49c27bfb70cb368d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:52.668602 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-4c5ee8c7-8e3d-4e45-bfb9-8f203a61cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:52.734025 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1275/2550] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:52.770765 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.772445 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:52.829973 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c348762-b5f5-44ae-a5d0-9f065700da32 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.834031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0604c-2262-481d-999f-7d238a21ee87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.834639 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0604c-2262-481d-999f-7d238a21ee87 service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.834904 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0604c-2262-481d-999f-7d238a21ee87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.835171 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0604c-2262-481d-999f-7d238a21ee87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.843205 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0604c-2262-481d-999f-7d238a21ee87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.844662 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0604c-2262-481d-999f-7d238a21ee87 service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.853824 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1276/2551] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:52.870615 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ee8c7-8e3d-4e45-bfb9-8f203a61cef3 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:52.928553 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:52.932858 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ee8c7-8e3d-4e45-bfb9-8f203a61cef3 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:52.938142 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:52.938777 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:52.939055 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:52.939329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] PATH_INFO: `/v3/users/fb7ac44555d744d49c27bfb70cb368d5` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:52.940498 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7ac44555d744d49c27bfb70cb368d5)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:52.942963 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:52.989694 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7ac44555d744d49c27bfb70cb368d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.001718 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7ac44555d744d49c27bfb70cb368d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.002380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7ac44555d744d49c27bfb70cb368d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.009762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7ac44555d744d49c27bfb70cb368d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.017335 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7ac44555d744d49c27bfb70cb368d5'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.023890 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1276/2552] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:32:52 2022] DELETE /identity/v3/users/fb7ac44555d744d49c27bfb70cb368d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:32:53.028440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:53.028722 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:53.028991 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:53.036928 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:32:53.048858 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:32:53.050067 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:53.107908 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1277/2553] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:32:53.117467 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:53.118861 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:53.174858 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953e350-9405-4e75-a129-936980871a2d admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:53.178506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:53.179099 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:53.179372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:53.179637 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:53.188506 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:53.189921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:53.199628 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1277/2554] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:32:53.463947 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:32:53.469392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:32:53.487014 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf85c-5963-4896-8eae-5237adeea67f None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:32:53.493815 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:32:53.494806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:32:53.495177 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:32:53.495541 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] PATH_INFO: `/v3/projects/17efd28d21a34bed81fa4f8b4eaac04e` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:32:53.496802 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17efd28d21a34bed81fa4f8b4eaac04e)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:32:53.498782 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:32:53.547232 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17efd28d21a34bed81fa4f8b4eaac04e', 'request_id': 'req-5b263aab-05c8-48f0-a621-a43675904499'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.557259 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17efd28d21a34bed81fa4f8b4eaac04e', 'request_id': 'req-5b263aab-05c8-48f0-a621-a43675904499'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.563186 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17efd28d21a34bed81fa4f8b4eaac04e', 'request_id': 'req-5b263aab-05c8-48f0-a621-a43675904499'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.575455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] The token cache is being invalidate because project 17efd28d21a34bed81fa4f8b4eaac04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:32:53.575740 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.576418 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-5b263aab-05c8-48f0-a621-a43675904499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:32:53.577886 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1278/2555] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:32:53 2022] DELETE /identity/v3/projects/17efd28d21a34bed81fa4f8b4eaac04e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:33:02.046519 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:02.047930 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:02.060200 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64826b5-3da9-46dd-a1c9-563f798f869b service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:02.117743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:02.118372 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.118643 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.118940 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] PATH_INFO: `/v3/users/00f25028b1f44112a8f8ed93b24c127d` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.120346 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f25028b1f44112a8f8ed93b24c127d)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:02.122304 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:02.144282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:02.168089 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f25028b1f44112a8f8ed93b24c127d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.179911 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f25028b1f44112a8f8ed93b24c127d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.180577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f25028b1f44112a8f8ed93b24c127d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.187412 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f25028b1f44112a8f8ed93b24c127d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.194889 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f25028b1f44112a8f8ed93b24c127d'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.201053 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1278/2556] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:33:02 2022] DELETE /identity/v3/users/00f25028b1f44112a8f8ed93b24c127d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:33:02.205492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.205999 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.206100 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.232169 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:02.233385 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:02.292244 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1279/2557] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:02.302494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:02.303896 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:02.359976 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2262367-cc97-460c-bbdc-879041d3d237 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:02.364129 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05f5c9-4d10-4a21-89b5-9718c20eebad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:02.364762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f5c9-4d10-4a21-89b5-9718c20eebad service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.365038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f5c9-4d10-4a21-89b5-9718c20eebad service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.365313 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f5c9-4d10-4a21-89b5-9718c20eebad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.374582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05f5c9-4d10-4a21-89b5-9718c20eebad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:02.375999 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05f5c9-4d10-4a21-89b5-9718c20eebad service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:02.385964 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1279/2558] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:33:02.655980 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:02.657406 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:02.670576 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05def0-287e-4fe5-945d-899981ac27b9 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:02.675271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:02.676544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.676874 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.677152 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] PATH_INFO: `/v3/projects/a99fae3350634a698293df70ed2b881a` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.679044 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99fae3350634a698293df70ed2b881a)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:02.681004 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:02.705776 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:02.729789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99fae3350634a698293df70ed2b881a', 'request_id': 'req-b8d6a003-a64a-46f8-979e-bea27ded8a54'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.736743 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99fae3350634a698293df70ed2b881a', 'request_id': 'req-b8d6a003-a64a-46f8-979e-bea27ded8a54'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.743032 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99fae3350634a698293df70ed2b881a', 'request_id': 'req-b8d6a003-a64a-46f8-979e-bea27ded8a54'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.754557 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] The token cache is being invalidate because project a99fae3350634a698293df70ed2b881a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:33:02.754840 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.755460 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-b8d6a003-a64a-46f8-979e-bea27ded8a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:33:02.757010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1280/2559] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:33:02 2022] DELETE /identity/v3/projects/a99fae3350634a698293df70ed2b881a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:33:02.761840 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833bcd-2476-4ce9-b9b2-b9d59d52c35b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.762119 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833bcd-2476-4ce9-b9b2-b9d59d52c35b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.762385 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15833bcd-2476-4ce9-b9b2-b9d59d52c35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.782062 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-15833bcd-2476-4ce9-b9b2-b9d59d52c35b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:02.783245 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-15833bcd-2476-4ce9-b9b2-b9d59d52c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:02.840824 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1280/2560] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:02.849009 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9c9db-b5d2-48b9-8a9e-fd6883ca09e0 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.849307 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9c9db-b5d2-48b9-8a9e-fd6883ca09e0 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.849580 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9c9db-b5d2-48b9-8a9e-fd6883ca09e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.869713 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-89c9c9db-b5d2-48b9-8a9e-fd6883ca09e0 None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:02.870896 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-89c9c9db-b5d2-48b9-8a9e-fd6883ca09e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:02.879848 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1281/2561] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:02.884261 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833bcd-2476-4ce9-b9b2-b9d59d52c35b None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:02.893762 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15833bcd-2476-4ce9-b9b2-b9d59d52c35b None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:02.898337 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:02.898929 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.899205 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.899469 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.900102 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 27 13:33:02.900425 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:02.901993 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:02.913702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1281/2562] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Oct 27 13:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 27 13:33:02.917971 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9c9db-b5d2-48b9-8a9e-fd6883ca09e0 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:02.928144 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9c9db-b5d2-48b9-8a9e-fd6883ca09e0 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:02.932860 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:02.933448 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:02.933727 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:02.933997 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:02.934626 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:02.936535 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:02.962967 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1282/2563] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:33:02 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:33:02.967069 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:02.977665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34437d62-644b-478f-a1bd-fca958d7fda0 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:03.003899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:03.035941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:03.036584 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:03.036856 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:03.037118 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:03.037747 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:03.039626 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:03.060808 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1282/2564] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:33:02 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:33:03.065281 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:03.075544 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da815f5-e64f-4dca-bf02-23136c203322 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:03.080526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:03.081117 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:03.081395 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:03.081665 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:03.082825 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:03.084422 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:03.094153 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1283/2565] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:33:03.098692 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:03.108483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e762-8dda-4cb8-aa37-32ee61d8aaad admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:03.112975 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4a4d5-2f66-4776-b75f-c42ba0c04131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:03.113580 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4a4d5-2f66-4776-b75f-c42ba0c04131 admin admin] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:03.113854 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4a4d5-2f66-4776-b75f-c42ba0c04131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:03.114122 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4a4d5-2f66-4776-b75f-c42ba0c04131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:03.114749 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4a4d5-2f66-4776-b75f-c42ba0c04131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:03.116364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4a4d5-2f66-4776-b75f-c42ba0c04131 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:03.127109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1283/2566] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:33:03.131224 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:03.141402 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0277618-e1b3-43f9-b86c-10f19be3a2d5 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:03.146606 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:03.147247 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] REQUEST_METHOD: `PUT` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:03.147526 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:03.147801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] PATH_INFO: `/v3/projects/a470328245084347ac78821d1a0184f7/users/1d5a8fcbaf9543aca96dea1ce9de3a46/roles/111b31616dc54e43a9000ee72a3ac5b8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:03.162843 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a470328245084347ac78821d1a0184f7, user_id=1d5a8fcbaf9543aca96dea1ce9de3a46, role_id=111b31616dc54e43a9000ee72a3ac5b8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:03.165502 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:03.178016 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1284/2567] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:33:03 2022] PUT /identity/v3/projects/a470328245084347ac78821d1a0184f7/users/1d5a8fcbaf9543aca96dea1ce9de3a46/roles/111b31616dc54e43a9000ee72a3ac5b8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:33:03.183073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc8388-ba67-42db-9157-70763154e75b None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:03.183449 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc8388-ba67-42db-9157-70763154e75b None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:03.183724 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc8388-ba67-42db-9157-70763154e75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:03.202431 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-18cc8388-ba67-42db-9157-70763154e75b None None] MFA Rules not processed for user `299712f6a5bb49b986da988b00e6dbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:03.203636 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-18cc8388-ba67-42db-9157-70763154e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:03.261737 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1284/2568] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:03.271753 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:03.273500 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:03.329265 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46384483-451d-4606-8b8a-1e65fa71567a admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:03.333440 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:03.334043 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:03.334344 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:03.334616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:03.343480 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:03.344984 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:03.354888 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1285/2569] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:33:07.996437 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8233be-ebaa-40d1-9014-6e6f40af1088 None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:07.997002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8233be-ebaa-40d1-9014-6e6f40af1088 None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:07.997002 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8233be-ebaa-40d1-9014-6e6f40af1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:08.027635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-fc8233be-ebaa-40d1-9014-6e6f40af1088 None None] MFA Rules not processed for user `1d5a8fcbaf9543aca96dea1ce9de3a46`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:08.029020 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-fc8233be-ebaa-40d1-9014-6e6f40af1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:08.093901 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1285/2570] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:08.098741 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:08.108386 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545278d2-5b04-4f3a-8477-ff2f866b509f service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:08.113655 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:08.114246 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:08.114552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:08.114822 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:08.115455 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:08.117392 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:08.144667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1286/2571] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Oct 27 13:33:08 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 27 13:33:08.148653 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8233be-ebaa-40d1-9014-6e6f40af1088 None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:08.159085 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8233be-ebaa-40d1-9014-6e6f40af1088 None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:08.191922 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:08.216430 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:08.217049 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:08.217368 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:08.217640 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] PATH_INFO: `/v3/users` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:08.218279 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:08.220177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:08.250281 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1286/2572] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Oct 27 13:33:08 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 27 13:33:08.255536 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:08.266162 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72b74d-f857-4b6e-9c5e-5bbc8ae0e014 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:08.271362 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:08.272055 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:08.272364 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:08.272641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:08.273285 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:08.274893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:08.286636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1287/2573] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:33:08.290877 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:08.301483 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149e22f-5836-4a79-8d9b-f3dc0d68fcca admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:08.306174 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:08.306823 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:08.307104 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:08.307381 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] PATH_INFO: `/v3/projects/07f26c9c2cbf4c7c8ff7eaa0baf3f2bd/users/fb74133fee7245018fbcef00704223d1/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:08.322571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f26c9c2cbf4c7c8ff7eaa0baf3f2bd, user_id=fb74133fee7245018fbcef00704223d1, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:08.325257 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:08.339624 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1287/2574] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:33:08 2022] PUT /identity/v3/projects/07f26c9c2cbf4c7c8ff7eaa0baf3f2bd/users/fb74133fee7245018fbcef00704223d1/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:33:08.343555 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:08.354986 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9f52-98f9-42dc-b12e-5a2ddecdcb79 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:08.367601 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:08.418433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad9a83-7dcb-42d9-98be-f5decd4e946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:08.420752 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad9a83-7dcb-42d9-98be-f5decd4e946d admin admin] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:08.421011 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad9a83-7dcb-42d9-98be-f5decd4e946d admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:08.421289 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad9a83-7dcb-42d9-98be-f5decd4e946d admin admin] PATH_INFO: `/v3/roles` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:08.421938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad9a83-7dcb-42d9-98be-f5decd4e946d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:08.424472 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad9a83-7dcb-42d9-98be-f5decd4e946d admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:08.438173 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1288/2575] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Oct 27 13:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 27 13:33:08.442847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:08.453687 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d0249-7155-4361-abc4-1a14f3d42996 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:08.458977 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:08.459646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:08.460063 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:08.460393 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] PATH_INFO: `/v3/projects/07f26c9c2cbf4c7c8ff7eaa0baf3f2bd/users/fb74133fee7245018fbcef00704223d1/roles/33f912ad05b44e36a696ef6aa86733be` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:08.462576 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f26c9c2cbf4c7c8ff7eaa0baf3f2bd, user_id=fb74133fee7245018fbcef00704223d1, role_id=33f912ad05b44e36a696ef6aa86733be)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:08.465529 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:08.476986 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1288/2576] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Oct 27 13:33:08 2022] PUT /identity/v3/projects/07f26c9c2cbf4c7c8ff7eaa0baf3f2bd/users/fb74133fee7245018fbcef00704223d1/roles/33f912ad05b44e36a696ef6aa86733be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:33:13.306206 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662f31f-56c2-403a-af80-55fbf7621782 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:13.307070 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662f31f-56c2-403a-af80-55fbf7621782 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:13.307283 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662f31f-56c2-403a-af80-55fbf7621782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:13.341547 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-3662f31f-56c2-403a-af80-55fbf7621782 None None] MFA Rules not processed for user `fb74133fee7245018fbcef00704223d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:13.343024 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-3662f31f-56c2-403a-af80-55fbf7621782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:13.427738 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1289/2577] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:13.442899 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:13.444743 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:13.527658 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab11ad-a748-47d8-b540-696e350c54c6 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:13.533704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7960f-f5ee-49ef-9bd7-ecd08701161a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:13.534357 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7960f-f5ee-49ef-9bd7-ecd08701161a service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:13.534635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7960f-f5ee-49ef-9bd7-ecd08701161a service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:13.534920 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7960f-f5ee-49ef-9bd7-ecd08701161a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:13.604907 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7960f-f5ee-49ef-9bd7-ecd08701161a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:13.606517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7960f-f5ee-49ef-9bd7-ecd08701161a service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:13.616388 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1289/2578] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:33:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:33:14.728608 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f0895-11c6-42b5-98d1-eb4b20956788 None None] REQUEST_METHOD: `POST` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:14.729231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f0895-11c6-42b5-98d1-eb4b20956788 None None] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:14.729231 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f0895-11c6-42b5-98d1-eb4b20956788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:14.750036 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.auth.core [None req-e79f0895-11c6-42b5-98d1-eb4b20956788 None None] MFA Rules not processed for user `1d5a8fcbaf9543aca96dea1ce9de3a46`. Rule list: `[]` (Enabled: `True`). {{(pid=81442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:14.751271 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e79f0895-11c6-42b5-98d1-eb4b20956788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:14.759731 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1290/2579] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:23.612282 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:24.744884 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=58 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:36.822414 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063d296-afa7-484f-ba74-aee9f8afd8af None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:36.822935 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063d296-afa7-484f-ba74-aee9f8afd8af None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:36.823038 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063d296-afa7-484f-ba74-aee9f8afd8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:36.846149 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-e063d296-afa7-484f-ba74-aee9f8afd8af None None] MFA Rules not processed for user `a41fc97ca6ea4a1c8201f068b26e6888`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:33:36.847466 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-e063d296-afa7-484f-ba74-aee9f8afd8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:36.926751 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1290/2580] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:33:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:33:46.908433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=25 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:33:49.366308 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f0895-11c6-42b5-98d1-eb4b20956788 None None] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:49.426316 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f0895-11c6-42b5-98d1-eb4b20956788 None None] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:49.430636 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:49.431342 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:49.431624 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:49.431898 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:49.442854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:49.444313 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:49.455429 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1291/2581] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:33:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:33:50.906665 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063d296-afa7-484f-ba74-aee9f8afd8af None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:50.926383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063d296-afa7-484f-ba74-aee9f8afd8af None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:50.933468 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:50.934300 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:50.934773 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:50.935386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:50.938448 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:50.952696 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:50.954851 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:50.966533 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1291/2582] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:33:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:33:53.368963 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:33:53.370469 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:53.382166 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd39dd0-8630-457e-9614-ce261341716e service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:33:53.385803 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:33:53.386393 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:33:53.386667 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:33:53.386937 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:33:53.388610 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:33:53.403683 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:33:53.405122 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:33:53.415793 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1292/2583] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:33:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:34:00.961406 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:34:03.411013 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=32 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:35:07.894716 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:35:07.904183 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def93486-13cc-492f-821b-749e0d1eca8c service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:35:07.907879 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8311e-083b-44f1-9009-51f805be019f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:35:07.908512 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8311e-083b-44f1-9009-51f805be019f service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:07.908787 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8311e-083b-44f1-9009-51f805be019f service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:07.909146 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8311e-083b-44f1-9009-51f805be019f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:07.911496 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-b5e8311e-083b-44f1-9009-51f805be019f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:07.921350 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8311e-083b-44f1-9009-51f805be019f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:35:07.922765 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8311e-083b-44f1-9009-51f805be019f service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:35:07.931485 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1292/2584] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:35:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:35:17.927417 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=6 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:35:40.069806 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:35:40.071250 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:40.128061 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e97bd9-650d-4817-80c2-7b11e1d0f6e3 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:35:40.131607 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80a57dd6e99c4cac9e6bcd1413afd5c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:35:40.132198 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:40.132495 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:40.132758 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:40.134627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:40.145552 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:35:40.146946 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:35:40.156322 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1293/2585] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:35:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:35:41.327982 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] REQUEST_METHOD: `POST` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:41.328542 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:41.328642 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:41.347928 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.auth.core [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] MFA Rules not processed for user `a41fc97ca6ea4a1c8201f068b26e6888`. Rule list: `[]` (Enabled: `True`). {{(pid=81441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 27 13:35:41.349143 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:41.356700 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1293/2586] 10.222.0.48 () {58 vars in 958 bytes} [Thu Oct 27 13:35:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 27 13:35:41.677640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:35:41.687348 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f496-77c8-4ada-9500-c516ca6d50da service cinder] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:35:41.690854 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:35:41.691436 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:41.691705 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:41.691966 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:41.693680 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:41.704187 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:35:41.705627 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:35:41.714534 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1294/2587] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:35:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:35:42.407400 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:35:42.409428 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:42.480086 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22370436-f0af-4e8b-9016-9baf345d6d7e None None] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:35:42.485043 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:35:42.485646 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:42.485921 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:42.486187 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:42.487912 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:42.499556 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:35:42.501153 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:35:42.511655 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1294/2588] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:35:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 27 13:35:45.204774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:35:45.207145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:45.231163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff36d6b-293b-4be2-8eda-78e7fb858f43 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:35:45.236757 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:35:45.237365 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:45.237640 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:45.237906 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:45.239642 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:45.253103 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:35:45.254690 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:35:45.265641 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1295/2589] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:35:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:35:52.506769 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=35 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:35:55.260769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=38 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:35:55.447979 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:35:55.463738 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2032a55-0a61-43da-92b4-d5da57817297 service placement] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:35:55.470474 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:35:55.471522 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:55.471819 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:55.472093 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:55.473892 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:55.495520 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:35:55.497704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:35:55.512404 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1295/2590] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:35:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:35:59.601677 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:35:59.615981 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef113c93-65d2-4593-a0c4-b826ba3dc8ca service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:35:59.622282 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d142bc3a9e4b3badefe713c7405329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:35:59.622894 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:35:59.623176 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:35:59.623447 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:35:59.625329 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:35:59.638300 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:35:59.639890 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:35:59.651812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1296/2591] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:35:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:36:02.836941 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:36:02.846394 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3153ef-965c-4b8e-88e2-ca7c5ff100c7 service nova] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:36:02.850039 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:36:02.850717 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:36:02.851054 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:36:02.851387 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:36:02.853399 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:36:02.864073 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:36:02.865530 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:36:02.875139 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1296/2592] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:36:02.951421 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:36:02.960919 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc43e94-b43b-4bf9-a008-9c6f07d23995 service nova] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:36:02.964561 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:36:02.965167 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:36:02.965438 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:36:02.965703 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:36:02.967417 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:36:02.980933 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:36:02.982336 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:36:02.995956 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1297/2593] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:36:12.870101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=12 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:36:12.989742 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=12 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:37:29.450885 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:29.463318 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c8c29-c6b1-4b10-8389-1d9b577ed26c service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:29.470433 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:29.471035 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:29.471315 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:29.471582 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:29.473364 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:29.485791 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:29.487443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:29.497949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1297/2594] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:37:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:37:33.871087 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:33.872317 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:33.887134 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef462d-22b5-47ff-8085-935f3c03fb26 service neutron] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:33.892616 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:33.893234 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:33.893514 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:33.893785 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:33.895568 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:33.907675 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:33.909304 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:33.922747 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1298/2595] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:37:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:37:37.108422 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:37.182489 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8012-6501-4b6b-a6c3-0986ea28f574 service placement] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:37.187048 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:37.187670 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:37.187949 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:37.188221 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] PATH_INFO: `/v3/users/1d5a8fcbaf9543aca96dea1ce9de3a46` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:37.189494 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d5a8fcbaf9543aca96dea1ce9de3a46)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:37.191386 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:37.240571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d5a8fcbaf9543aca96dea1ce9de3a46'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.253177 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d5a8fcbaf9543aca96dea1ce9de3a46'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.253844 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d5a8fcbaf9543aca96dea1ce9de3a46'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.261443 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d5a8fcbaf9543aca96dea1ce9de3a46'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.269244 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d5a8fcbaf9543aca96dea1ce9de3a46'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.275648 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1298/2596] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:37:37 2022] DELETE /identity/v3/users/1d5a8fcbaf9543aca96dea1ce9de3a46 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:37:37.460420 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:37.471550 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e86cb-bef7-4372-887e-09a36c520609 service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:37.528312 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:37.528885 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:37.529163 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:37.529435 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] PATH_INFO: `/v3/projects/a470328245084347ac78821d1a0184f7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:37.530533 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a470328245084347ac78821d1a0184f7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:37.532456 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:37.578297 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a470328245084347ac78821d1a0184f7', 'request_id': 'req-67532329-d583-45a7-95b1-69d91d6d65b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.585204 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a470328245084347ac78821d1a0184f7', 'request_id': 'req-67532329-d583-45a7-95b1-69d91d6d65b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.590769 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a470328245084347ac78821d1a0184f7', 'request_id': 'req-67532329-d583-45a7-95b1-69d91d6d65b0'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.603721 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] The token cache is being invalidate because project a470328245084347ac78821d1a0184f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:37:37.604010 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.604674 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.606200 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1299/2597] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:37:37 2022] DELETE /identity/v3/projects/a470328245084347ac78821d1a0184f7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:37:37.610088 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:37.611475 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:37.624352 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af68035-eed3-43f4-baef-42143053002f admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:37.682561 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:37.683203 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:37.683482 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:37.683770 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] PATH_INFO: `/v3/users/fb74133fee7245018fbcef00704223d1` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:37.684927 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb74133fee7245018fbcef00704223d1)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:37.686883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:37.692165 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:37:37.733830 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb74133fee7245018fbcef00704223d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.745435 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb74133fee7245018fbcef00704223d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.746623 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb74133fee7245018fbcef00704223d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.753403 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb74133fee7245018fbcef00704223d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.761095 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb74133fee7245018fbcef00704223d1'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:37.767306 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1299/2598] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:37:37 2022] DELETE /identity/v3/users/fb74133fee7245018fbcef00704223d1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:37:37.936578 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:37.938114 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:37.951433 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67532329-d583-45a7-95b1-69d91d6d65b0 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:38.008031 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:38.008685 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:38.009214 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:38.009483 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] PATH_INFO: `/v3/projects/07f26c9c2cbf4c7c8ff7eaa0baf3f2bd` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:38.010588 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f26c9c2cbf4c7c8ff7eaa0baf3f2bd)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:38.013096 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:38.033774 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:37:38.060468 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f26c9c2cbf4c7c8ff7eaa0baf3f2bd', 'request_id': 'req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:38.067872 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f26c9c2cbf4c7c8ff7eaa0baf3f2bd', 'request_id': 'req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:38.073720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f26c9c2cbf4c7c8ff7eaa0baf3f2bd', 'request_id': 'req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:38.085817 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] The token cache is being invalidate because project 07f26c9c2cbf4c7c8ff7eaa0baf3f2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:37:38.086103 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:38.086725 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:38.088182 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1300/2599] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:37:37 2022] DELETE /identity/v3/projects/07f26c9c2cbf4c7c8ff7eaa0baf3f2bd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:37:47.765948 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:37:48.082790 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:37:51.352263 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:51.353786 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:51.411577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c5cc0-5ca0-4569-bc96-1f42cd2c3273 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:51.465587 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cba7b6049e41488f5b2f7c61a0500a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:51.466211 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:51.466486 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:51.466753 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:51.468501 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:51.530380 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:51.531780 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:51.541101 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1300/2600] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:37:54.870247 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:54.873062 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:54.955780 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec97e6-0f34-4a0c-a0b8-c9a1a6e8250c admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:54.960994 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:54.961603 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] REQUEST_METHOD: `GET` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:54.961881 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:54.962154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:54.963875 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:54.975563 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:54.977109 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:54.987850 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1301/2601] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:37:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 27 13:37:57.808544 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:57.810061 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:57.823329 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357020-1bb6-4e86-9f93-09f5d6e74ef6 service neutron] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:57.828070 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:57.828776 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:57.829059 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:57.829356 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] PATH_INFO: `/v3/users/a41fc97ca6ea4a1c8201f068b26e6888` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:57.830490 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41fc97ca6ea4a1c8201f068b26e6888)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:57.832517 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:57.878514 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41fc97ca6ea4a1c8201f068b26e6888'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:57.888702 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:37:57.891120 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41fc97ca6ea4a1c8201f068b26e6888'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:57.892215 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41fc97ca6ea4a1c8201f068b26e6888'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:57.899402 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41fc97ca6ea4a1c8201f068b26e6888'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:57.906635 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41fc97ca6ea4a1c8201f068b26e6888'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:57.912536 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1301/2602] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:37:57 2022] DELETE /identity/v3/users/a41fc97ca6ea4a1c8201f068b26e6888 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:37:58.081605 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:58.092691 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284642a-f0fa-4d0b-803b-26278347c0ae service placement] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:58.148635 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:58.149254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:58.149524 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:58.149789 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] PATH_INFO: `/v3/projects/f291a3a8c0394e928aee758baa470ca7` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:58.150870 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f291a3a8c0394e928aee758baa470ca7)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:58.152762 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:58.198622 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f291a3a8c0394e928aee758baa470ca7', 'request_id': 'req-ed81de72-c8fe-4b14-bce1-faafe7196034'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.205042 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f291a3a8c0394e928aee758baa470ca7', 'request_id': 'req-ed81de72-c8fe-4b14-bce1-faafe7196034'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.210520 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f291a3a8c0394e928aee758baa470ca7', 'request_id': 'req-ed81de72-c8fe-4b14-bce1-faafe7196034'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.221814 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] The token cache is being invalidate because project f291a3a8c0394e928aee758baa470ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:37:58.222094 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.222720 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.224154 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1302/2603] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:37:58 2022] DELETE /identity/v3/projects/f291a3a8c0394e928aee758baa470ca7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:37:58.229109 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:58.230571 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:58.244371 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3b55-b4a9-4b43-beb3-bec4ccd6b9ac admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:58.306792 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:58.307336 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:58.307613 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:58.307883 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] PATH_INFO: `/v3/users/bb199ca4c05e4c7fbf759abf4d23050c` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:58.309033 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb199ca4c05e4c7fbf759abf4d23050c)` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:58.310932 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:58.357541 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb199ca4c05e4c7fbf759abf4d23050c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.369704 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb199ca4c05e4c7fbf759abf4d23050c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.370383 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb199ca4c05e4c7fbf759abf4d23050c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.377480 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb199ca4c05e4c7fbf759abf4d23050c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.384740 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.notifications [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb199ca4c05e4c7fbf759abf4d23050c'} {{(pid=81441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.390956 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1302/2604] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Oct 27 13:37:58 2022] DELETE /identity/v3/users/bb199ca4c05e4c7fbf759abf4d23050c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:37:58.830599 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Authenticating user token {{(pid=81442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:37:58.832108 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.fernet_utils [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:37:58.844801 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81de72-c8fe-4b14-bce1-faafe7196034 admin admin] Validating token access rules against request {{(pid=81442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:37:58.862276 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:37:58.903492 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '299712f6a5bb49b986da988b00e6dbe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f354da2936f649f78cdcb89056983d15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:37:58.904145 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:37:58.904446 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:37:58.904712 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] PATH_INFO: `/v3/projects/5c38edf003e74119a853a203be7a2fd8` {{(pid=81442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:37:58.905812 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c38edf003e74119a853a203be7a2fd8)` {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:37:58.907687 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] RBAC: Authorization granted {{(pid=81442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:37:58.953977 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c38edf003e74119a853a203be7a2fd8', 'request_id': 'req-171c0e08-a369-4daf-a699-fe137073b602'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.961136 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c38edf003e74119a853a203be7a2fd8', 'request_id': 'req-171c0e08-a369-4daf-a699-fe137073b602'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.966598 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c38edf003e74119a853a203be7a2fd8', 'request_id': 'req-171c0e08-a369-4daf-a699-fe137073b602'} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.977968 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] The token cache is being invalidate because project 5c38edf003e74119a853a203be7a2fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 27 13:37:58.978254 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.978893 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG keystone.notifications [None req-171c0e08-a369-4daf-a699-fe137073b602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 27 13:37:58.980397 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: [pid: 81442|app: 0|req: 1303/2605] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Oct 27 13:37:58 2022] DELETE /identity/v3/projects/5c38edf003e74119a853a203be7a2fd8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 27 13:38:08.387434 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG dbcounter [-] [81441] Writing DB stats keystone:DELETE=12,keystone:SELECT=60,keystone:INSERT=2 {{(pid=81441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:38:08.974938 dv-cinder-862432-3-iscsi devstack@keystone.service[81442]: DEBUG dbcounter [-] [81442] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 27 13:43:15.185296 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Authenticating user token {{(pid=81441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 27 13:43:15.186847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:43:15.328616 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162d484-eb98-4d9c-83fc-4fa753157045 admin admin] Validating token access rules against request {{(pid=81441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 27 13:43:15.388847 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638a9b2-9a8d-4e6d-bcb6-a1a1388bc968 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a578acf1e0cc4505a03c1b587b381b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aba5f43c2f94c27a9d4152aa186289e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 27 13:43:15.389450 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638a9b2-9a8d-4e6d-bcb6-a1a1388bc968 service placement] REQUEST_METHOD: `GET` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 27 13:43:15.389726 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638a9b2-9a8d-4e6d-bcb6-a1a1388bc968 service placement] SCRIPT_NAME: `/identity` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 27 13:43:15.390124 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638a9b2-9a8d-4e6d-bcb6-a1a1388bc968 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 27 13:43:15.391871 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.fernet_utils [None req-0638a9b2-9a8d-4e6d-bcb6-a1a1388bc968 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 27 13:43:15.403984 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638a9b2-9a8d-4e6d-bcb6-a1a1388bc968 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 27 13:43:15.405577 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638a9b2-9a8d-4e6d-bcb6-a1a1388bc968 service placement] RBAC: Authorization granted {{(pid=81441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 27 13:43:15.416456 dv-cinder-862432-3-iscsi devstack@keystone.service[81441]: [pid: 81441|app: 0|req: 1303/2606] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Oct 27 13:43:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)